feat(imitation): add profile-driven dual-arm collection

[TomCC7]

Contribution path

• PRD: Manipulation SDK preview, discussion #3923.
• Dual-arm integration layer, stacked on feat(imitation): add CLI learning workflow #3931.
• Shared fixes live below this PR: webcam backend fix(camera): select V4L2 for Linux device paths #4159, robot construction/joints fix(openyam): align learning stack with canonical model #3853, source interpretation feat(imitation): add native OpenYAM collection #3854, and viewer identity feat(imitation): add CLI learning workflow #3931.

Problem

• Dual OpenYAM collection needs two wrist RGB streams and a canonical 14-D state/action contract.
• The collection setup needs externally accessible Viser without changing the ordinary teleop default.
• Shared fixes should live in their owning layers, not be introduced and replaced in this integration PR.

Solution

• Add DUAL_OPENYAM_COLLECTION and the ordinary dual-openyam-quest-collection Blueprint.
• Compose the robot-local teleop construction helper with two cameras, the existing recorder factory, and episode monitor.
• Configure collection with ViserVisualizationConfig(host="0.0.0.0"). Preserve model, kinematics, control tasks, and the base teleop's local-only default.
• Remove manual Blueprint atom/kwargs extraction. No new generic Blueprint configuration API.
• Use canonical joints and explicit source meaning from the parent layers.
• Keep this PR to five files: profile, Blueprint, integration tests, generated registry entry, and usage documentation.
• RGB only. ABC integration, dual-arm policy rollout, and generalized backend remapping are outside this PR.

API shape

DUAL_OPENYAM_COLLECTION
  observation.images.left_wrist  <- left_wrist_image
  observation.images.right_wrist <- right_wrist_image
  observation.state              <- coordinator_joint_state.position
  action                         <- applied_joint_position_command.position

• Images: 640x480 RGB. Dataset rate: 30 Hz; left-wrist anchor; 20 ms alignment tolerance.
• Joint order: left arm 1-6, right arm 1-6, left gripper, right gripper.
• CollectionFeature.source_kind: "snapshot" for images/measured state; "joint_position_updates" for accepted commands.
• Snapshots use nearest-frame alignment. Commands are reconstructed causally from recording history, including across episode boundaries; omitted joints retain their targets.
• Shared-source projections must agree on source kind. Missing initial joints and malformed updates fail validation. There is no legacy sampling alias or automatic recording migration.

dual_openyam_quest_collection = autoconnect(
    build_dual_openyam_webxr(
        visualization=ViserVisualizationConfig(host="0.0.0.0"),
    ),
    # Camera producers are configured by the robot Blueprint.
    left_wrist_camera,
    right_wrist_camera,
    collection_recorder(profile=DUAL_OPENYAM_COLLECTION),
    EpisodeMonitorModule.blueprint(instance_name="episodes"),
)

• This snippet abbreviates the camera declarations and remappings; the runnable Blueprint is in dual_openyam/blueprints/learning_collection.py.
• Profiles have no registry. Blueprint discovery selects the setup; the profile name is recording metadata.
• Additional cameras require profile features and matching producers, not a recorder subclass.
• Recordings contain schema.json plus recording.mcap or recording.db. Capture remains unaligned and native-rate; inspection and preparation share the offline evaluation pipeline.

How to Test

With two OpenYAM arms, Quest, and two wrist RGB cameras:

uv run dimos run dual-openyam-quest-collection --daemon --recorder.recording recordings/fold-001 --recorder.format mcap --episodes.task "fold the towel" --controlcoordinator.left-can-port follower_l --controlcoordinator.right-can-port follower_r --left-wrist.hardware.camera-index /dev/video0 --right-wrist.hardware.camera-index /dev/video2
uv run dimos imitation collect

uv run dimos imitation inspect recordings/fold-001
uv run dimos imitation prepare recordings/fold-001 --output datasets/fold-001
uv run dimos imitation visualize datasets/fold-001 --episode 0

• Use new recording and output directories. Repeat capture with --recorder.format sqlite.
• Open http://ROBOT_HOST:8095 on a trusted network, or pass --manipulationmodule.visualization.host 127.0.0.1.
• Q detaches controls; recording and hardware continue. Support both arms before dimos stop.
• Cleanup verification: 224 focused tests, six registry checks, mypy on 10 changed production files, Ruff, and commit hooks passed.
• Existing fold-001, evaluated with the updated profile without changing its saved schema: 250 + 171 frames, zero filled, 16.55 ms maximum alignment error.
• Both camera features decode for all 421 frames. Every reconstructed action exactly matches independent raw-MCAP history and the existing LeRobot export.
• Original recording/schema checksums are unchanged. No new hardware capture or interactive viewer validation was performed for this cleanup.

AI assistance

• OpenAI Codex assisted extensively with implementation, tests, documentation, and stack cleanup.
• Updated by OpenAI Codex (agent).

Checklist

• I have read and approved the CLA.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

@@                      Coverage Diff                       @@
##           cc/feat/robot-learning-sdk    #3942      +/-   ##
==============================================================
+ Coverage                       78.37%   79.72%   +1.34%     
==============================================================
  Files                            1530     1533       +3     
  Lines                          144292   144355      +63     
  Branches                        12319    12319              
==============================================================
+ Hits                           113095   115088    +1993     
+ Misses                          27903    25887    -2016     
- Partials                         3294     3380      +86     

Components	Coverage Δ
Tests	95.35% <100.00%> (+1.91%)	⬆️

Flag	Coverage Δ
OS-ubuntu-24.04-arm	75.59% <100.00%> (+0.01%)	⬆️
OS-ubuntu-latest	76.33% <100.00%> (?)
Py-3.11	76.33% <100.00%> (?)
Py-3.12	75.59% <100.00%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
dimos/robot/all_blueprints.py	100.00% <ø> (ø)
...ors/dual_openyam/blueprints/learning_collection.py	100.00% <100.00%> (ø)
...ipulators/dual_openyam/blueprints/test_learning.py	100.00% <100.00%> (ø)
dimos/robot/manipulators/dual_openyam/learning.py	100.00% <100.00%> (ø)

... and 41 files with indirect coverage changes

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[greptile-apps]

Confidence Score: 2/5

Not safe to merge until recording always has complete action targets and remote robot controls are restricted to trusted users.

Findings

1.  Initialize every action joint ▶
2.   Protect remote robot controls ▶

Summary

This change adds a dual-OpenYAM Quest collection profile, blueprint, registry entry, focused configuration coverage, and usage documentation. Two demonstrated failures require attention: recordings can be rejected when sparse accepted commands do not yet include both gripper targets, and the collection blueprint exposes interactive robot controls to reachable unauthenticated clients.

_{Reviews (1) · Last reviewed commit: "test: avoid resolving model assets in co..."}

[greptile-apps]

Initialize every action joint

The 14-joint action is reconstructed from accepted sparse command updates, but recording can start after an arms-only 12-joint command. Strict preparation then has no targets for left_arm/gripper and right_arm/gripper, emits no samples, and rejects the episode. Seed a complete 14-joint command state or prevent recording until every canonical joint has an accepted target.

Knowledge Base Used: Robot control platform

Artifacts

Evidence from the check

• The authored script builds strict profile-backed streams with partial and complete accepted commands and runs the real preparation/projection APIs, providing the reproducible check.

Command output from the check

• The partial-command run supplied 12 arm joints and shows strict projection rejecting the episode because both gripper targets are absent, confirming the failure.

Command output from the check

• The complete-command run supplied all 14 canonical joints and shows strict preparation emitting four valid 14-element action samples, establishing the control result.

_{Ran code and verified through T-Rex}

[greptile-apps]

Protect remote robot controls

This collection blueprint binds Viser to 0.0.0.0, while the visualization path has no authentication setting. A client that can reach the service can load enabled Plan and Execute controls, allowing unauthenticated access to interactive robot planning and execution. Keep the service loopback-only by default, or require an authenticated and authorized network boundary before enabling non-loopback access.

How this was verified: A live all-interface listener rendered enabled Plan and Execute controls with no configured authentication gate.

Artifacts

Evidence from the check

• This executed Python check reads the changed blueprint and Viser implementation, then starts a real all-interface Viser listener; it is the exact test input used to establish the exposure.

Command output from the check

• The successful command output records the all-interface blueprint setting, direct host forwarding, missing auth config fields, real `*:8097` Viser listener, and successful connection; the takeaway is that the configured Viser service is reachable without a configured authentication gate.

Evidence from the check

• This minimal real Viser server creates the same Plan and Execute panel controls used for the Chromium comparison; it is the executed UI fixture.

Evidence from the check

• This Playwright script launches Chromium, verifies visible and enabled Plan and Execute controls, captures a poster frame, and records the UI video; it is the exact browser automation input.

▶ Recording of the check

• Chromium renders the Viser panel when the server is explicitly bound to `127.0.0.1:8096`, showing the Plan and Execute controls remain local-only in the comparison condition.

Loopback-only Viser panel with Plan and Execute controls

• The Chromium poster frame shows the Viser manipulation panel and its Plan and Execute controls under the loopback-only comparison condition.

Command output from the check

• The successful command log shows Viser listening at `127.0.0.1:8096` and Playwright finding visible, enabled Plan and Execute controls; the takeaway is that the baseline scope is loopback-only.

▶ Recording of the check

• Chromium renders the Viser panel after binding the server to `0.0.0.0:8095`, visibly showing the interactive Plan and Execute controls in the network-exposed condition.

All-interface Viser panel with Plan and Execute controls

• The Chromium poster frame shows the Viser manipulation panel with the enabled Plan and Execute controls in the all-interface condition.

Command output from the check

• The successful command log records `LISTEN 0.0.0.0:8095`, Viser `auth=none`, and Playwright-visible/enabled Plan and Execute controls; the takeaway is that reachable clients can receive this unauthenticated interactive panel.

_{Ran code and verified through T-Rex}