Projects · Part II
ATLAS
Learned exploration over a persistent belief graph, trained and tested in simulation and now transferring to the drone.
Trained and tested in simulation. Current drone transfer underway; earlier iterations flight-tested.
01 — The system
What it does
The robot enters an unmapped indoor space with one forward-facing camera and has to cover it under a time budget without collision. Motion is treated as solved: a geometric planner proposes only routes it has verified against the occupancy map the robot has built. The open problem is choice, not motion, so the learned layer decides among certified options and may abstain, but never commands motion itself. ATLAS is built in PyTorch and fully trained and tested in simulation. Current work transfers the policy to the drone, building on earlier flight-tested iterations.
Engineering ownership
My contribution
- Designed the persistent belief graph and learned selection of geometrically admitted exploration routes.
- Trained and tested the policy in simulation; tested earlier iterations on the drone and am transferring the current policy.
Learned choice, with geometric checks
- 01RememberPlaces, open ends, obligations
- 02Propose & certifyGeometrically checked routes
- 03RankLearned option selection
- 04ExecuteSelected route through the planner
02 — Evidence
How the agent decides
The state is a belief graph, not a map
ATLAS carries a persistent causal belief graph rather than a snapshot of the occupancy grid. It has three kinds of node: places the robot has been, open ends where its knowledge of the space stops, and obligations it has taken on and not yet discharged. Every edge is keyed by a certified route and by the hash of that route's certificate, so an edge exists only where a verified way through exists, and it stops existing when that certificate does. Nodes carry an 18-dimensional feature vector, edges a 7-dimensional one.
Two views of the same surroundings
A graph neural network reads that structure, and its output is fused with ten-channel body-aligned occupancy crops taken at two scales: 8 m across at 0.125 m per cell for what is immediately around the robot, and 24 m at 0.25 m for the wider region it sits in. The crops are body-aligned, so the representation turns with the robot instead of with the world frame.
Proposing and ranking the options
A transformer over candidate tokens proposes and ranks up to eight options at each decision, through four heads: proposal, choice, count and criterion. It is trained by behaviour cloning against a classical teacher whose routes are themselves certified, so the learned policy starts from decisions that were admissible by construction.
Scoring an option before committing to it
Each option is scored by a distributional effect model, which predicts a distribution over what the option would produce rather than a single expected value, and by a horizon-3 ICBO MPC that chains those predictions three decisions deep. Both score against a reward that was registered before the experiments: new observed area, arrival in a region, and resolution of an obligation, charged against elapsed time, control effort, and revisiting ground that yields nothing.
The learned layer cannot steer
All route certification and all collision authority sit outside the network by construction. The learned model re-orders options that an independent certifier has already admitted, and it may put none of them forward. There is no path from the network to the actuators that does not pass through the certifier.
The control layer underneath
Below the option layer sits a depth-only control policy: a temporal transformer over the same dual-scale occupancy, with no colour and no semantic input. It is trained first by counterfactual advantage-weighted behaviour cloning, then by constrained SMDP-PPO with duration-aware GAE, which credits temporally extended options by how long they actually ran, under a Lagrangian budget on collision cost.
Design commitments
Certification creates, learning selects.
- Options are judged against one another, so two routes leading through the same opening are not treated as genuine alternatives.
- Actions are temporally extended, because passing through a doorway is locally expensive and locally uninformative, and everything of value happens afterwards. The critic credits them by their duration.
- Unfinished work is a node type rather than a heuristic: an obligation stays in the graph until it is discharged, so regions get completed and exhausted ones get abandoned on evidence rather than on a timer.
- Every edge is keyed to a certified route and its certificate hash, so the graph cannot express a way through that has not been verified.
- The reward is registered before the experiments, so the objective cannot be tuned to the result.
- No learned component is permitted to command motion. It re-orders certified options or refuses them.
What is being tested
If the state of exploration is a persistent belief graph of places, open ends and obligations rather than a snapshot of the map, if options over that graph are scored by a distributional model of their effects under a reward fixed in advance, and if every route the learned layer can rank has already been certified by something outside it, then the agent should finish the regions it starts, leave them when the evidence says they are exhausted rather than when time runs out, and stay safe where it is ignorant.