Ruthwik/feat/dimsim object annotations - #4208
Draft
ruthwikdasyam wants to merge 4 commits into
Draft
ruthwikdasyam wants to merge 4 commits into
ruthwikdasyam wants to merge 4 commits into
Conversation
Add an "Object labels + boxes" toggle to the DimSim viewer that snapshots
world-axis-aligned bounds and authored names for identified scene assets,
rendered in a separate overlay scene so physics and robot RGB/depth/LiDAR
captures are untouched.
Export the same snapshot as vision_msgs.Detection3DArray in the DimOS Z-up
"world" frame via SceneClient.get_object_detections() and
export_object_detections(). Each detection carries the stable asset id,
the authored label as a hypothesis with score 1.0, and an identity-oriented
bbox; the .bin export is one LCM wire payload, publishable with
LCMTransport("/detections_3d", Detection3DArray).
Include walls in the object snapshot: structure nodes named like "wall-north" or "yard-wall-east" (and SceneClient.add_wall meshes) are bounded under their node name, outside the assets group, so the overlay and the Detection3DArray export cover 87 assets + 20 walls in the apartment. export_object_detections() writes a readable JSON view for ".json" paths (id, label, center_xyz, size_xyz, orientation_xyzw per detection) and the typed LCM payload otherwise.
Reference Detection3DArray export of the apartment scene as JSON: 107 entries (87 identified assets + 20 baked walls) with id, label, center_xyz, size_xyz and orientation_xyzw in the ROS Z-up "world" frame.
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## main #4208 +/- ##
==========================================
+ Coverage 79.53% 79.55% +0.02%
==========================================
Files 1492 1494 +2
Lines 141368 141547 +179
Branches 12120 12128 +8
==========================================
+ Hits 112436 112614 +178
Misses 25511 25511
- Partials 3421 3422 +1
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 2 files with indirect coverage changes 🚀 New features to boost your workflow:
|
Reads misc/DimSim/scenes/apartment/object_detections.json and renders the 107 boxes as an interactive matplotlib 3D plot (rotate/zoom/pan), with --objects / --walls filters and a --save PNG mode.
ruthwikdasyam
force-pushed
the
ruthwik/feat/dimsim-object-annotations
branch
from
September 17, 2026 23:51
578e5dc to
ae2a22d
Compare
ruthwikdasyam
added a commit
that referenced
this pull request
Sep 17, 2026
Replace the manifest-derived scene (real centres, invented 1 m squares) with DimSim's own annotation snapshot from PR #4208: 107 objects and walls with real extents, already in the ROS world frame odometry uses. Copied verbatim (misc/DimSim/scenes/apartment/object_detections.json at 7fd0e2f); the live SceneClient.get_object_detections() emits the same schema, so going live later is a one-line change in run(). load_scene() now reads that schema and filters geometrically instead of by word list: boxes whose bottom clears the robot's 0.74 m capsule top are overhead (door headers, wall cabinets, the TV); footprints contained in another kept footprint are clutter (books on shelves, plates on carts, bedding, a chair under its table). 107 -> 15 walls + 33 objects. Room bounds come from the outer walls. The second live run ended pressed against the real couch, 0.69 m before my approximate box even began, and scored 0.54 because arrival measured to the couch's centre, which is inside the couch. The grader now measures to the goal box edge; that run re-scores 0.85 (test added). The state gains goal_box so "reached" has something concrete to mean. Step criteria drop compass words: the scene's wall labels put "wall-east" at +y, which contradicts "+y = north" in the same prompt. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 task
Draft
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contribution path
Problem
Solution
How to Test
AI assistance
Checklist