Bavantha Udugama 0000-0002-1932-692X

Projects · Part I

M2H

Four dense tasks in one model, replacing two separate networks that did not agree with each other.

2025 · published

The scene graph forming over one loop of the ITC second floor, from the first frame to loop closure. Playback is 10x real time. Live RGB, predicted semantics, and predicted depth run down the right.

01 — The system

What it does

Window-based cross-task attention on a DINOv2-small backbone, predicting depth, semantics, surface normals, and edges together so that each task can use what the others found.

Engineering ownership

My contribution

  • Developed a shared model for depth, semantics, surface normals and edges using cross-task attention.
  • Integrated the learned predictions into monocular scene graph construction and evaluated mapping performance.
System sketch

One image, four connected prediction tasks

  1. 01
    InputOne RGB image
  2. 02
    EncodeDINOv2-small backbone
  3. 03
    Share informationWindow-based cross-task attention
  4. 04
    Predict togetherDepth, semantics, normals, edges
Conceptual model overview. The experimental figures below show the actual predictions and mapping output.

02 — Evidence

The scene graph forming over one loop of the ITC second floor, from the first frame to loop closure. Playback is 10x real time. Live RGB, predicted semantics, and predicted depth run down the right.Watch the full clip

ITC building corridor

A corridor in the ITC building seen by the monocular camera: doors on the right, artwork on the walls, an orange floor receding into the distance.RGB input
Predicted depth for the same corridor frame, rendered as a colour field where blue is near and red is the far end of the corridor.Predicted depth
Predicted semantic labels for the same corridor frame, with flat colour regions for wall, floor, ceiling, and door.Predicted semantics
One frame from the ITC corridor sequence: camera image, predicted depth, predicted labels. Pick a layer, or drag to blend between them.

Measured

Key numbers

  1. 61.54 mIoU and 0.4196 RMSE on NYUDv2
  2. 0.11 m ITC mean error at 30 fps
  3. 81 M parameters, 488 GFLOPs

Every figure here appears in the thesis results registry with its source location. Conditions are stated with the number.

Paper

  1. M2H: Multi-Task Learning with Efficient Window-Based Cross-Task Attention for Monocular Spatial PerceptionPublished, IEEE/RSJ IROS 2025

Code

  1. M2Hmodel, training, pretrained weights