Skip to content

Projects · Part I

Mono-Hydra++

The current system. One camera, one IMU, a hierarchical 3D scene graph, and a perception loop that feeds back into state estimation.

2025 to 2026 · under review

Mono-Hydra++ system overview. Scene and map views are illustrative.

01 — The system

What it does

Predicted sparse depth enters the visual-inertial odometry as metric anchor factors. Semantic masks keep dynamic and unreliable regions out of the estimator. The poses that come back out align the next dense predictions in time. Perception feeds the estimator and the estimator feeds perception.

Engineering ownership

My contribution

Developed the perception–estimation loop: sparse learned depth factors, semantic masking and temporal alignment.
Integrated perception, odometry and scene graph construction on Jetson and built and flight-tested the custom drone.

System sketch

From motion estimates to refined poses and scene graphs

01
SenseOne camera + IMU
02
PredictLearned depth + semantic masks
03
Refine posesVisual, motion, IMU and sparse-depth constraints
04
MapHierarchical 3D scene graph

RVIO2 poses → align depth and labels across frames

RVIO2 supplies motion estimates. Local graph refinement combines motion, visual, IMU and sparse-depth constraints before sending refined poses to Hydra.

Hardware × software integration

Configure the robot around one camera and an IMU.

The hardware, inference runtime and mapping backend have to work as one system. This is the configuration behind my onboard mapping work.

The same quadrotor from directly above, showing the square carbon frame, four red propellers inside their cage, the orange lithium battery strapped across the centre, and a cooling fan on the compute board. — From above: battery across the centre, compute and its cooling fan beneath.

Built and flight-tested

Carry the mapping stack into the air.

I built the custom drone and integrated perception, odometry and scene graph construction on its Jetson. The carbon frame, propeller cage, battery placement and onboard compute are visible in the hardware photographs.

Sensing: One RGB camera + IMU
Compute: Jetson Orin NX
Inference: ONNX → TensorRT FP16
Mapping: RVIO2 + local refinement + Hydra

01 / Inputs

Sense motion and appearance

RGB images provide appearance; the IMU provides inertial measurements. RVIO2 supplies motion estimates for the mapping pipeline.

RGB + IMU → motion estimates

02 / Compute

Run learned perception onboard

The Jetson Orin NX carries the perception workload. Models are exported through ONNX and run with TensorRT FP16; predicted depth and semantic labels feed the mapping stack.

Images → depth + semantic labels

03 / Integration

Align predictions and refine poses

Relative poses align predictions across frames. Selected depth samples join visual, motion and IMU constraints in local graph refinement.

Aligned predictions → refined poses

04 / Output

Construct a structured map

Hydra receives refined poses and loop-closure proposals, maintains the geometric map and builds the hierarchical 3D scene graph.

Geometry → objects, places and rooms

Watch the platform fly.

Take-off and a pass down the corridor. Everything the mapping runs on is carried onboard.

Inspect the onboard pose estimate.

Depth-backed visual-inertial odometry on the drone's Jetson, walked down and back up the stairs at home. The label switches between MOVING and ZUPT STATIONARY: when the platform is still, zero-velocity updates hold the estimate in place instead of letting it drift. Predicted depth, tracked features, and predicted semantics are the panels on the left.

The flight clip shows the physical platform; the mapping recordings show software output. They are separate recordings. Perception throughput is not an end-to-end mapping or flight-control rate.

Architecture / Mono-Hydra++

Trace a camera frame through the mapping stack.

RGB and IMU measurements become depth, semantic labels, motion estimates, refined poses and a hierarchical map.

Mono-Hydra++ system overview: RGB and IMU feed perception and odometry; aligned depth supports local graph refinement and volumetric fusion; Hydra builds a hierarchical scene graph. — Mono-Hydra++ system overview. Scene and map views are illustrative. Open full-size ↗

01 / Inside Mono-Hydra++

Align predictions before fusing the map.

A pixel in the previous frame does not refer to the same scene point after the camera moves. RVIO2 relative poses warp historical depth and labels into the current view. The nearest sample is retained at each pixel; accepted history is fused with the current prediction using depth medians and label voting.

The output is aligned depth and labels for local graph refinement and volumetric fusion.

Previous depth and labels are back-projected, transformed by relative camera pose and reprojected into the current image; validity checks precede depth median and label voting. — Pose-aware temporal alignment. Illustrative scene views show how history is aligned and combined with the current prediction. Open full-size ↗

02 / Inside Mono-Hydra++

Turn selected depth samples into pose constraints.

Aligned depth is sampled at tracked keypoints. Depth validity, motion, reprojection and semantic checks select the samples that can become depth factors. These enter local graph refinement alongside visual tracks, RVIO2 motion and IMU constraints.

Refined poses and loop-closure proposals are passed to Hydra, which optimises the map and updates the scene graph.

Tracked keypoints sample aligned depth; validity, motion, reprojection and semantic checks select depth factors for local graph refinement. — Sparse depth-factor selection. Scene views are illustrative. Open full-size ↗

03 / Inside Mono-Hydra++

Constrain optical-axis depth in the local graph.

The depth measurement is camera-frame distance along the optical axis. A depth factor connects a camera pose and a tracked landmark; visual, motion and IMU factors provide complementary constraints.

The illustration omits velocity and bias variables to make the pose–landmark relationship easier to read.

Optical-axis depth relates a camera pose to a landmark; the local factor graph combines visual, IMU, motion and depth constraints. — Depth-factor geometry and local factor graph. Velocity and bias variables are omitted in this illustration. Open full-size ↗

02 — Evidence

The custom quadrotor on a wooden table, seen from the side. A carbon fibre frame with a protective cage around four red propellers, the flight controller and wiring visible on top, and the compute board mounted underneath. — Carbon fibre frame, caged propellers, and the Jetson Orin NX underneath that runs perception, odometry, mapping, and scene graph construction.

The same quadrotor from directly above, showing the square carbon frame, four red propellers inside their cage, the orange lithium battery strapped across the centre, and a cooling fan on the compute board. — Carbon fibre frame, caged propellers, and the Jetson Orin NX underneath that runs perception, odometry, mapping, and scene graph construction.

A 3D scene graph over the ITC building second floor. A single building node at the top connects down to a room node, then to a dense band of place nodes following the corridor loop, then to object nodes scattered along the walls, and finally to the reconstructed mesh of the floor. — The layered scene graph over the ITC second floor. It runs from one building node down through a room node, a band of place nodes following the corridor loop, and object nodes along the walls. Counts by category are in the legend.

Reconstruction of the ITC second floor produced from the embedded operating point on the Jetson Orin NX at reduced input resolution. — The same floor rebuilt from the Jetson operating point at 256x192. Mean error is 0.22 m here against 0.08 m at full resolution.

Measured

Key numbers

0.08 m ITC 2nd floor mean error
0.033 m calibrated ATE on 7-Scenes
44.96 % semantic mesh mIoU on ScanNet

Every figure here appears in the thesis results registry with its source location. Conditions are stated with the number.

Paper

Mono-Hydra++: Real-Time Monocular Scene Graph Construction with Multi-Task Learning for 3D Indoor MappingUnder review at ISPRS Journal of Photogrammetry and Remote Sensing

Code