feat(habitat): HSSD scene object and wall ground truth as Detection3DArray

[ruthwikdasyam]

Contribution path

• Linked issue or discussion: feat(evals): add QA suites for 18 Habitat scenes #4195 (Habitat QA suites for these scenes)

Problem

The Habitat QA suites in #4195 describe the ten HSSD scenes in prose; nothing machine-readable says where the objects and walls are.

Solution

Ground-truth 3D boxes for all ten HSSD QA scenes, checked in as misc/habitat/ground_truth/hssd/<scene_id>.json: the JSON view of a vision_msgs.Detection3DArray, one Detection3D per object and per wall in the ROS Z-up world frame (Habitat (x, y, z) -> (-z, -x, y)).

• dimos/simulation/habitat/hssd_ground_truth.py generates them offline, no habitat-sim needed: it reads scenes/<id>.scene_instance.json placements and the stage/object GLBs with trimesh, measures tight world AABBs from mesh vertices, labels objects from HSSD semantics/objects.csv (category as class_id, product name as a second hypothesis; the object-config semantic_id lexicon is wrong for these assets and is not used), and boxes each wall_<n> group of the stage. The two top-level merged geometry_wall* meshes per stage are skipped: they are hundreds of disconnected quads spanning the whole building.
• dimos/simulation/object_detections.py holds the box-to-Detection3DArray conversion and JSON view; both box corners go through the axis mapping and are re-sorted, so the sign flips are safe.
• misc/habitat/README.md documents the format, ids, labels, frame, and the regeneration command.
• pyproject.toml: the ground-truth JSON directory is exempted from the 75 KB large-files hook (largest scene is 202 KB, 1.1 MB in total).

Per scene: 95/423/173/77/164/279/331/64/268/264 objects and 22/31/14/13/18/14/37/11/26/27 walls. Boxes are world-axis-aligned, so rotated objects and diagonal walls get loose boxes; orientation is always the identity.

How to Test

uv run pytest dimos/simulation
HSSD_DATASET_CONFIG=<hssd-hab>/hssd-hab.scene_dataset_config.json \
  uv run python -m dimos.simulation.habitat.hssd_ground_truth --scene 102344193

Checked against the real dataset: the laptop and TV anchor points recorded during QA authoring fall inside their boxes, all walls measure 2.8 m tall, and 84 of 95 scene-1 objects match habitat-sim's own transformed AABBs exactly or tighter (the other 11 are duplicate chairs and stools the check script paired in the wrong order; their offsets equal the spacing between duplicates).

AI assistance

Claude Code with Fable 5.1 implemented the exporter, tests and docs from an approved plan; the author chose the offline trimesh approach, the label scheme and the wall-by-node-name rule.

Checklist

• I have read and approved the CLA.

[codecov]

Codecov Report

❌ Patch coverage is 98.13520% with 8 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
dimos/simulation/habitat/hssd_ground_truth.py	95.50%	4 Missing and 4 partials ⚠️

@@            Coverage Diff             @@
##             main    #4211      +/-   ##
==========================================
+ Coverage   79.54%   79.58%   +0.04%     
==========================================
  Files        1492     1498       +6     
  Lines      141416   142124     +708     
  Branches    12122    12178      +56     
==========================================
+ Hits       112487   113109     +622     
- Misses      25508    25587      +79     
- Partials     3421     3428       +7     

Components	Coverage Δ
Tests	95.37% <100.00%> (+0.01%)	⬆️

Flag	Coverage Δ
OS-ubuntu-24.04-arm	75.25% <98.13%> (+0.11%)	⬆️
OS-ubuntu-latest	76.07% <98.13%> (+0.10%)	⬆️
Py-3.10	75.85% <98.13%> (+0.10%)	⬆️
Py-3.11	76.05% <98.13%> (+0.10%)	⬆️
Py-3.12	76.06% <98.13%> (+0.10%)	⬆️
SelfHosted-Linux	36.63% <33.79%> (-0.02%)	⬇️
SelfHosted-macOS	36.97% <33.79%> (-0.03%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
dimos/simulation/habitat/test_hssd_ground_truth.py	100.00% <100.00%> (ø)
dimos/simulation/object_detections.py	100.00% <100.00%> (ø)
dimos/simulation/test_object_detections.py	100.00% <100.00%> (ø)
dimos/simulation/habitat/hssd_ground_truth.py	95.50% <95.50%> (ø)

... and 7 files with indirect coverage changes

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.