Bavantha Udugama 0000-0002-1932-692X

Projects · Part I

M2H-MX

The perception front end that improved the map, and the ablation showing where the improvement came from.

2026 · published

The whole pipeline on ScanNet scene 0000. M2H-MX predicts depth and semantics from the image stream, Mono-Hydra++ consumes those predictions inside the visual-inertial odometry and the mapping backend, and the hierarchical scene graph is built in real time. Playback is 16x.

01 — The system

What it does

DINOv3 features adapted with LoRA, decoded through register-gated Mamba blocks at multiple scales, with directed cross-task refinement between depth and semantics. Exported to ONNX and TensorRT FP16 for the drone.

Engineering ownership

My contribution

  • Developed the multi-task perception model and evaluated its predictions inside the mapping pipeline.
  • Exported and optimised embedded inference through ONNX and TensorRT FP16 on Jetson Orin NX.
System sketch

Perception designed for a running mapping system

  1. 01
    InputOne RGB image
  2. 02
    Adapt featuresDINOv3 + LoRA
  3. 03
    Decode & refineRegister-gated Mamba + cross-task refinement
  4. 04
    Use on the robotDepth & semantics for mapping
A simplified view of the model and its deployment path. ONNX / TensorRT FP16 is the embedded inference route.

02 — Evidence

The whole pipeline on ScanNet scene 0000. M2H-MX predicts depth and semantics from the image stream, Mono-Hydra++ consumes those predictions inside the visual-inertial odometry and the mapping backend, and the hierarchical scene graph is built in real time. Playback is 16x.Watch the full clip

Indoor, NYUDv2

An indoor living room: a person seated on a patterned sofa, a table lamp, a wooden side table, an armchair, and framed pictures on the wall.RGB input
Predicted depth for the same living room, as a smooth colour field where the near armchair and sofa stand out from the far wall.Predicted depth
Predicted semantic labels for the same living room, with flat colour regions for sofa, chair, lamp, table, floor, wall, and pictures.Predicted semantics
Indoor frame, NYUDv2. Metric depth and semantic labels from one RGB image. Pick a layer, or drag to blend between them.

Outdoor, Cityscapes

A city street seen from a vehicle: parked and moving cars, a yellow DHL lorry, road markings, traffic signs, trees, and buildings.RGB input
Predicted depth for the same street, with the road surface graded from near to far and vehicles resolved as separate depth bodies.Predicted depth
Predicted semantic labels for the same street, with distinct colours for road, cars, vegetation, buildings, poles, and traffic signs.Predicted semantics
The same model outdoors, Cityscapes: 82.28 mIoU, 3.15 points over the strongest baseline. Pick a layer, or drag to blend between them.
M2H-MX architecture diagram. A DINOv3 backbone with LoRA adaptation feeds token reassembly and a feature pyramid, then register-gated Mamba blocks at each scale, task adaptors, a cross-task mixer, multi-scale convolutional attention, and finally the task heads.
M2H-MX. The backbone stays mostly frozen and LoRA adapts the later blocks, so the compute goes into the decoder and the cross-task exchange rather than into retraining the features.

Measured

Key numbers

  1. 65.60 mIoU and 0.3800 RMSE on NYUDv2
  2. 6.91 cm average ScanNet ATE, down from 17.59 cm
  3. 25.53 fps on Jetson Orin NX 16GB at 256x192

Every figure here appears in the thesis results registry with its source location. Conditions are stated with the number.

Paper

  1. M2H-MX: Multi-Task Semantic and Geometric Perception for Real-Time Monocular 3D Scene Graph ConstructionAccepted, ICRA 2026 SRRA Workshop

Code

  1. m2h_mxmodel and training