feat(habitat): top-down 2D projection of HSSD ground truth as Detection2DArray - #4214
Draft
ruthwikdasyam wants to merge 3 commits into
Draft
ruthwikdasyam wants to merge 3 commits into
ruthwikdasyam wants to merge 3 commits into
Conversation
…on2DArray top_down() flattens a Detection3DArray to a metric Detection2DArray: each box's floor footprint as a rectangle with the same x/y center and size, theta 0, same id, labels and header, in the ROS Z-up "world" frame. The 3D center stays in each hypothesis pose; only the height is dropped. Boxes whose footprint spans at least 90% of the scene extent on both axes (a floor slab or roof) are left out and logged. The HSSD exporter now also writes <scene_id>.top_down.json next to the 3D file for all ten QA scenes (2351 boxes, nothing dropped), and misc/habitat/render_top_down.py draws each one as a reference SVG: walls filled grey, objects outlined with their category inside.
Codecov Report❌ Patch coverage is
@@ Coverage Diff @@
## ruthwik/feat/hssd-ground-truth #4214 +/- ##
==================================================================
- Coverage 79.62% 79.61% -0.01%
==================================================================
Files 1498 1496 -2
Lines 142024 141994 -30
Branches 12162 12162
==================================================================
- Hits 113086 113055 -31
- Misses 25514 25516 +2
+ Partials 3424 3423 -1
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 6 files with indirect coverage changes 🚀 New features to boost your workflow:
|
The wall_<n> stage groups only hold the exterior shell; the interior partitions live in the merged top-level wallTop mesh that the exporter used to skip. Walls now come from every ceiling-height geometry_wallTop face in the stage: touching collinear faces merge while the union stays wall-thin (so L and T junctions stay separate), near-coincident thick pieces (stacked bay-window tops) merge into one, and each segment is extruded down to the stage floor. Ids are wall_<k> by footprint position, label "wall". Regenerates the ten scenes' 3D, top-down and SVG files: walls are now 21/44/9/12/17/14/36/14/22/35 and show every room partition.
Walls now come from cutting the stage geometry 20 cm above its floor: whatever is solid there is a wall segment, a doorway is a gap between two segments a robot can pass through, and door or window headers are not boxed. Cut lines merge into thin segments (perpendicular walls stay separate) and each segment is extruded up to the top of the geometry standing in its footprint, so half walls keep their height. Ids stay wall_<k> by footprint position, label "wall". The top-down SVG draws walls in the same outline style as objects; a doorway is simply where no box is. Regenerates the ten scenes' 3D, top-down and SVG files: walls are now 32/67/14/16/25/20/59/20/35/42.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contribution path
Problem
The 3D ground truth from #4211 is the full box set; for map-style consumers (robot navigation) we also want the same scene flattened to a top-down 2D view, still labeled, as typed data rather than a picture. Reviewing that view showed #4211's walls were incomplete (interior partitions missing) and, once recovered from wall tops, closed over doorways, which a navigation consumer must not see as solid.
Solution
dimos/simulation/object_detections.py:top_down(Detection3DArray) -> Detection2DArray. Each box becomes aDetection2DwhoseBoundingBox2Dis the floor footprint (same x/y center and size,theta = 0) in the same ROSworldframe, with the sameid, labels and header; the 3D center is kept in each hypothesis pose, only the height is dropped. Boxes whose footprint spans at least 90% of the scene extent on both axes (a floor slab or roof) are left out and logged (none in these scenes). Oriented boxes raise rather than project wrong. Plus the matching JSON view and writer.dimos/simulation/habitat/hssd_ground_truth.py: also writes<scene_id>.top_down.jsonnext to the 3D file, and rebuilds the walls from a horizontal slice of the stage 20 cm above the floor. Whatever is solid there is a wall segment; a doorway is a gap between two segments a robot can pass through; door and window headers are not boxed. Cut lines merge into thin segments (perpendicular walls stay separate, stacked frames collapse), and each segment is extruded up to the top of the geometry standing in its footprint, so half walls keep their height. Ids arewall_<k>by footprint position, labelwall. Boxes at L and T junctions overlap by the neighbour's thickness.misc/habitat/render_top_down.py: standalone script drawing a*.top_down.jsonas a reference SVG (walls and objects in one outline style, category written inside objects that can hold text, 1 m grid, meters); a doorway is simply where no box is. SVG keeps the images as plain text; the JSONs and SVGs are exempted from the 75 KB hook inpyproject.toml.misc/habitat/README.mddocuments the 2D files, the wall rule and the renderer.Checked in for all ten scenes:
misc/habitat/ground_truth/hssd/<scene_id>.json(regenerated with the new walls),.top_down.jsonand.top_down.svg. Per scene: 95/423/173/77/164/279/331/64/268/264 objects and 32/67/14/16/25/20/59/20/35/42 walls. Boxes are world-axis-aligned, so rotated objects and diagonal walls get loose boxes; orientation is always the identity. Since the 3D files change here, the wall change can be moved down into #4211 if preferred.Example 2D entry:
{"id": "670c0caf8cb7df8466c675d7c91f7877840f9513@12", "label": "chair", "score": 1.0, "center_xy": [7.628, 2.069], "size_xy": [0.528, 0.603], "theta": 0.0, "labels": ["chair", "Siena 27 Inch Bar Stool, Blue"]}How to Test
Verified: every 2D entry across the ten scenes equals its 3D source's x/y center, size and labels; counts match; the rendered plans of scenes 102344193, 102344403 and 103997424_171030444 were eyeballed and show every room partition with open doorways (bedrooms, bathrooms, closets, laundry, garage). Wall thickness is 0.2 to 0.35 m in median per scene; the few thick "walls" are authored stage masses (bay windows, a built-in block).
AI assistance
Claude Code with Fable 5.1 implemented the projection, the slice-based walls, the renderer, tests and docs; the author chose the 2D type, the scene-covering filter, SVG over PNG, flagged the missing walls and specified the doorway behaviour.
Checklist