Skip to content

feat(imitation): add direct OpenYAM teach collection - #3921

Open
TomCC7 wants to merge 6 commits into
feat/openyam-lerobot-rolloutfrom
cc/feat/openyam-teach-data-collection
Open

TomCC7 wants to merge 6 commits into
feat/openyam-lerobot-rolloutfrom
cc/feat/openyam-teach-data-collection

Conversation

@TomCC7

@TomCC7 TomCC7 commented Sep 3, 2026

Copy link
Copy Markdown
Member

Contribution path

Problem

  • Operators need to collect OpenYAM demonstrations by moving the arm and gripper by hand, without Quest.
  • Teaching should be a robot preset over the shared recorder, not a separate collection framework or CLI.

Solution

  • Add openyam-teach-collection using the existing trajectory task's idle-hold behavior.
  • Configure zero position stiffness, gravity feed-forward, and arm damping (2, 2, 2, 0.5, 0.5, 0.5).
  • Add Damiao passive-gripper configuration: retain opening calibration, then continuously command zero stiffness, damping, and feed-forward torque while reading measured opening.
  • Define OPENYAM_TEACH_COLLECTION with measured seven-joint positions for both state and hand-guided action.
  • Reuse the recorder factory, recording directory, and episode-control Spec from feat(imitation): add native OpenYAM collection #3854.
  • Keep Quest collection unchanged. The operator TUI belongs in feat(imitation): add CLI learning workflow #3931; no dimos collect command or learning-profile wrapper is introduced here.

API shape

openyam-teach-collection
  ControlCoordinator: zero-stiffness arm + passive gripper
  wrist:              ordinary CameraModule
  recorder:           collection_recorder(OPENYAM_TEACH_COLLECTION)
  episodes:           EpisodeMonitorModule
  • Profile location: dimos.robot.manipulators.openyam.collection.
  • Both measured projections use source_kind="snapshot"; teaching actions are measurements, not partial command updates.
  • State and action project coordinator_joint_state.position; wrist RGB is the image observation.
  • Device and recording settings use ordinary module options. Episode RPCs use EpisodeControlSpec.

How to Test

With an arm and wrist RGB camera configured:

dimos --can-port follower_l run openyam-teach-collection --recorder.recording recordings/teach-001 --episodes.task "pick up the cube" --wrist.hardware.camera-index 0
  • Control episodes through EpisodeControlSpec; support the arm before runtime shutdown.
  • Confirm free hand movement of the calibrated gripper, suitable damping, and meaningful measured joint motion before collecting a dataset.
  • In the previous cleanup: 78 focused Damiao, teaching-Blueprint, and episode-monitor tests; 6 registry checks; scoped mypy passed.
  • Current review cleanup rebases this preset onto the source-kind contract without changing teaching behavior.
  • The previous cleanup did not rerun physical teaching. Existing datasets were left untouched. All commit checks passed except the teaching worktree's LFS-upload check, skipped because of an unrelated pre-existing dataset.

AI assistance

  • Review cleanup updated by OpenAI Codex (agent).
  • OpenAI Codex (GPT-5) assisted extensively with implementation, tests, documentation, and stack cleanup.

Checklist

  • I have read and approved the CLA.

@TomCC7 TomCC7 changed the title cc/feat/openyam teach data collection feat(imitation): add direct OpenYAM teach collection Sep 3, 2026
@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.91304% with 7 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...pulators/openyam/blueprints/learning_collection.py 86.36% 1 Missing and 2 partials ⚠️
dimos/hardware/whole_body/damiao/adapter.py 88.88% 1 Missing and 1 partial ⚠️
dimos/hardware/whole_body/damiao/test_adapter.py 97.50% 1 Missing ⚠️
...ors/openyam/blueprints/test_learning_collection.py 96.96% 0 Missing and 1 partial ⚠️
@@                      Coverage Diff                       @@
##           feat/openyam-lerobot-rollout    #3921    +/-   ##
==============================================================
  Coverage                         79.65%   79.66%            
==============================================================
  Files                              1519     1521     +2     
  Lines                            143269   143383   +114     
  Branches                          12237    12246     +9     
==============================================================
+ Hits                             114128   114229   +101     
- Misses                            25787    25809    +22     
+ Partials                           3354     3345     -9     
Components Coverage Δ
Tests 95.33% <97.26%> (-0.03%) ⬇️
Flag Coverage Δ
OS-ubuntu-24.04-arm 75.48% <93.91%> (+0.01%) ⬆️
OS-ubuntu-latest 76.25% <93.91%> (+0.01%) ⬆️
Py-3.10 76.03% <93.91%> (+0.01%) ⬆️
Py-3.11 76.23% <93.91%> (+0.01%) ⬆️
Py-3.12 76.24% <93.91%> (+0.01%) ⬆️
SelfHosted-Large 30.72% <34.78%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
dimos/hardware/whole_body/damiao/config.py 100.00% <100.00%> (ø)
dimos/robot/all_blueprints.py 100.00% <ø> (ø)
dimos/robot/manipulators/openyam/collection.py 100.00% <100.00%> (ø)
dimos/hardware/whole_body/damiao/test_adapter.py 98.48% <97.50%> (+0.53%) ⬆️
...ors/openyam/blueprints/test_learning_collection.py 96.96% <96.96%> (ø)
dimos/hardware/whole_body/damiao/adapter.py 96.28% <88.88%> (+1.84%) ⬆️
...pulators/openyam/blueprints/learning_collection.py 86.36% <86.36%> (ø)

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread dimos/cli/commands/collect.py Outdated
Comment thread dimos/control/tasks/teach_task/teach_task.py Outdated
@TomCC7
TomCC7 marked this pull request as ready for review September 4, 2026 02:18
@greptile-apps

greptile-apps Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This change adds a dashboard for controlling OpenYAM teach-data collection alongside Quest input. Testing reproduced a control race in dimos/cli/commands/collect.py: if Quest changes the recording state between dashboard polls, Space can save a take that the dashboard still shows as not recording, while D can decline to send the discard request and leave that take active. This should be corrected before merging.

Confidence Score: 4/5

Not safe to merge until dashboard episode commands are made consistent with the authoritative recording state.

A focused executable reproduction exercised both normal and stale-state control flows and observed an unintended save as well as a suppressed discard request.

Files Needing Attention: dimos/cli/commands/collect.py needs state-aware action dispatch; its interaction with dimos/imitation/collection/episode_monitor.py should preserve an atomic authoritative decision.

T-Rex T-Rex Logs

What T-Rex did

  • Created a focused reproduction source for the dashboard stale-status race.
  • Inspected the dashboard actions when cached and authoritative state agree, using the corresponding log.
  • Inspected the dashboard actions after authoritative state changes to recording between polls, using the corresponding log.
  • Validated contract behavior by comparing baseline and stale-state outcomes, noting the toggle/discard results and the claimed unintended save and suppressed discard.

View all artifacts

T-Rex Ran code and verified through T-Rex

Comments Outside Diff (1)

  1. General comment

    P1 Dashboard actions act on stale cached episode state

    • Bug
      • Between the 250 ms dashboard polls, Quest or another controller can change the monitor from idle to recording. If the dashboard still displays READY, Space invokes action_toggle_recording and sends toggle; the authoritative monitor resolves that toggle as save, ending and saving the remote recording. In the same stale-display state, D enters action_discard's local idle guard and returns without sending the discard RPC, leaving the remote recording active.
    • Cause
      • action_toggle_recording does not obtain fresh status before sending the state-dependent toggle RPC (collect.py:308-309), while action_discard bases whether to send its RPC solely on self._status (collect.py:311-316). The monitor independently resolves toggle against its current locked authoritative state (episode_monitor.py:177-191).
    • Fix
      • Refresh/validate authoritative state immediately before action dispatch, or replace the ambiguous toggle/local guard with server-side state-aware explicit commands that atomically express the intended UI action and return the resulting status. In particular, discard must not be suppressed solely by dashboard-cached state.

    T-Rex Ran code and verified through T-Rex

Reviews (1): Last reviewed commit: "refactor(imitation): simplify direct tea..." | Re-trigger Greptile

Comment thread dimos/cli/commands/collect.py Outdated
Comment on lines +308 to +316
def action_toggle_recording(self) -> None:
self._episode_command("toggle")

def action_discard(self) -> None:
if self._status.state != "recording":
self._message = "Nothing to discard. Start a take first."
self._refresh()
return
self._episode_command("discard")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Dashboard actions use stale episode state

The dashboard derives its commands from self._status, which is only updated during polling, while Quest can change the monitor state independently. If Quest starts recording after the last poll, the dashboard can still show idle: Space sends toggle, which the authoritative monitor resolves as save and unexpectedly ends the active take. In that same stale-idle state, D returns from the local guard without sending discard, leaving the recording active. Make the intended action state-aware at the authoritative monitor, or refresh and atomically validate status before dispatching it.

Artifacts

Focused dashboard stale-status race reproduction source

  • The executable harness imports the PR dashboard actions and runs them against the monitor's documented transition semantics, showing the exact baseline and stale-state cases.

Dashboard actions when cached and authoritative state agree

  • The executed baseline run shows toggle starting recording and discard issuing an RPC that increments the discard count, establishing the normal expected behavior.

Dashboard actions after authoritative state changes to recording between polls

  • The executed race run shows stale-idle Space saving the authoritative recording and stale-idle D making no discard RPC, proving the candidate bug.

View artifacts

T-Rex Ran code and verified through T-Rex

@TomCC7
TomCC7 force-pushed the cc/feat/openyam-teach-data-collection branch from 127ce28 to 1ef8eb8 Compare September 4, 2026 03:24
@TomCC7
TomCC7 force-pushed the cc/feat/openyam-teach-data-collection branch 2 times, most recently from 562643c to 6bca282 Compare September 8, 2026 23:42
@TomCC7
TomCC7 force-pushed the cc/feat/openyam-teach-data-collection branch from 6bca282 to 58b5b5c Compare September 8, 2026 23:55
@TomCC7
TomCC7 force-pushed the cc/feat/openyam-teach-data-collection branch from 58b5b5c to 7463ad0 Compare September 10, 2026 20:48
@TomCC7
TomCC7 removed this pull request from stack #3922 September 15, 2026 04:24
@TomCC7
TomCC7 added this pull request to stack #4160 September 15, 2026 04:24
@TomCC7
TomCC7 force-pushed the cc/feat/openyam-teach-data-collection branch 2 times, most recently from 2a69c7c to 9a7c673 Compare September 15, 2026 05:51
@TomCC7
TomCC7 force-pushed the cc/feat/openyam-teach-data-collection branch from 9a7c673 to 960058e Compare September 17, 2026 23:36
@TomCC7
TomCC7 force-pushed the cc/feat/openyam-teach-data-collection branch from 960058e to d62d515 Compare September 18, 2026 00:35
@TomCC7
TomCC7 force-pushed the cc/feat/openyam-teach-data-collection branch from d62d515 to 6fbe6c9 Compare September 18, 2026 00:39
@TomCC7
TomCC7 force-pushed the cc/feat/openyam-teach-data-collection branch from 6fbe6c9 to 59ad909 Compare September 18, 2026 01:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant