Skip to content

fix(openyam): align learning stack with canonical model - #3853

Merged
mustafab0 merged 13 commits into
mainfrom
fix/openyam-learning-baseline
Sep 18, 2026
Merged

mustafab0 merged 13 commits into
mainfrom
fix/openyam-learning-baseline

Conversation

@TomCC7

@TomCC7 TomCC7 commented Sep 1, 2026

Copy link
Copy Markdown
Member

Contribution path

Problem

  • Learning, teleoperation, and hardware must agree on OpenYAM joint names, home pose, and limits.
  • IK must not solve using joints that the selected controller cannot command.

Solution

  • Define canonical arm/gripper joint names in a hardware-independent module, shared by robot configuration and later collection/rollout presets.

  • Align single- and dual-arm home configurations with the canonical model.

  • Reduce Pink's model to controlled joints and apply configured velocity limits.

  • Add OpenYAM-specific teleop weighting and regression coverage.

  • Keep recorder, policy, and CLI concerns in later layers.

  • Own the dependency-free dual-arm joint declarations and import them directly from dual_openyam.joints; remove config reexports.

  • Add robot-local build_dual_openyam_webxr(visualization=...) to construct the full teleop graph without extracting Blueprint atom kwargs.

  • Load dual-arm hardware limits from the canonical model and report Viser execution acceptance/rejection.

  • Keep lazy model paths lazy during configuration copying and serialization introspection; preserve normal resolution on asset access.

  • Keep canonical-joint unit tests independent of model downloads and complete the pygame sequence type stub.

API shape

  • dimos.robot.manipulators.openyam.joints exports OPENYAM_ARM_JOINTS, OPENYAM_GRIPPER_JOINT, and OPENYAM_JOINTS.
  • dimos.robot.manipulators.dual_openyam.joints owns the ordered DUAL_OPENYAM_ARM_JOINTS, DUAL_OPENYAM_GRIPPER_JOINTS, and DUAL_OPENYAM_JOINTS declarations.
  • Robot configuration owns hardware/model setup; there is no separate learning-profile model.
  • build_dual_openyam_webxr(*, visualization: ManipulationVisualizationConfig = ViserVisualizationConfig()) -> Blueprint accepts the existing visualization config. Ordinary teleop remains on 127.0.0.1; callers can construct a graph with a different host. No generic Blueprint .configure() API.

How to Test

With OpenYAM and Quest configured:

dimos --can-port follower_l run teleop-webxr-openyam
  • In the previous cleanup, 22 focused OpenYAM/config tests passed; the final stack also passed 52 Pink/OpenYAM IK tests.

  • Current review cleanup: 40 focused baseline tests and mypy on five changed production files passed. Full-stack verification also covers the inherited Viser and dual-arm regressions.

  • Exercise weighted IK, home pose, and velocity limits on supported hardware before release. Hardware operation was not retested for this cleanup.

  • CI repair: 38 focused baseline tests and all 37 configuration-parser tests passed; lazy-copy tests cover unresolved/resolved paths and shared references. Scoped mypy passed.

AI assistance

  • Review cleanup updated by OpenAI Codex (agent).
  • OpenAI Codex (GPT-5) assisted extensively with implementation, tests, documentation, and stack cleanup.

Checklist

  • I have read and approved the CLA.

@codecov

codecov Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.45560% with 4 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...obot/manipulators/dual_openyam/blueprints/basic.py 88.23% 1 Missing and 1 partial ⚠️
dimos/robot/manipulators/openyam/teleop_ik.py 83.33% 1 Missing and 1 partial ⚠️
@@            Coverage Diff             @@
##             main    #3853      +/-   ##
==========================================
+ Coverage   79.53%   79.57%   +0.04%     
==========================================
  Files        1492     1498       +6     
  Lines      141368   141657     +289     
  Branches    12120    12130      +10     
==========================================
+ Hits       112436   112730     +294     
+ Misses      25511    25502       -9     
- Partials     3421     3425       +4     
Components Coverage Δ
Tests 95.36% <100.00%> (+0.01%) ⬆️
Flag Coverage Δ
OS-ubuntu-24.04-arm 75.15% <79.92%> (+0.01%) ⬆️
OS-ubuntu-latest 75.98% <86.48%> (+0.02%) ⬆️
Py-3.10 75.76% <86.48%> (+0.02%) ⬆️
Py-3.11 75.96% <86.48%> (+0.02%) ⬆️
Py-3.12 75.97% <86.48%> (+0.02%) ⬆️
SelfHosted-Large 30.61% <36.29%> (-0.02%) ⬇️
SelfHosted-Linux 36.69% <57.91%> (+0.02%) ⬆️
SelfHosted-macOS 37.02% <57.91%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...are/whole_body/dual_openyam_damiao/test_adapter.py 100.00% <100.00%> (ø)
dimos/manipulation/manipulation_module.py 75.12% <100.00%> (+0.18%) ⬆️
...os/manipulation/planning/kinematics/pink_solver.py 82.42% <100.00%> (+4.50%) ⬆️
...s/manipulation/planning/kinematics/test_pink_ik.py 99.23% <100.00%> (+0.01%) ⬆️
dimos/manipulation/visualization/viser/test_gui.py 98.25% <100.00%> (+0.06%) ⬆️
...bot/manipulators/dual_openyam/blueprints/teleop.py 100.00% <100.00%> (ø)
...manipulators/dual_openyam/blueprints/test_basic.py 100.00% <100.00%> (ø)
...anipulators/dual_openyam/blueprints/test_teleop.py 100.00% <100.00%> (ø)
dimos/robot/manipulators/dual_openyam/config.py 94.11% <100.00%> (-0.62%) ⬇️
dimos/robot/manipulators/dual_openyam/joints.py 100.00% <100.00%> (ø)
... and 12 more

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@TomCC7
TomCC7 marked this pull request as ready for review September 1, 2026 03:22
@greptile-apps

greptile-apps Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This change aligns OpenYAM planning and teleoperation with the canonical robot description, adds an explicit collection-ready home posture and velocity limits, reduces Pink IK models to commanded joints, and applies OpenYAM-specific Quest posture weighting.

The executed Pinocchio runtime check verified that controlled-joint reduction removes uncommanded joints, preserves the commanded joint mapping, and writes configured velocity limits to the correct Pinocchio velocity coordinates even when model coordinate order differs. No reportable defect was found in those exercised behaviors.

T-Rex validation blocked

Canonical OpenYAM model loading could not be exercised because the required data/.lfs/yam_description.tar.gz asset is an unresolved Git LFS pointer and the git-lfs tool is missing. Quest solver construction and posture-weight execution could not be exercised because the isolated runtime is missing the turbojpeg package.

Confidence Score: 5/5

No merge-blocking defect was found in the executed controlled-joint reduction and velocity-limit paths.

The completed runtime check passed for the highest-risk joint-mapping and velocity-coordinate behavior, and no final defect comments were produced. Canonical model and Quest execution remain unexercised because their required local dependencies are unavailable.

Files Needing Attention: The canonical-asset paths in dimos/robot/manipulators/openyam/config.py and the Quest solver path in dimos/robot/manipulators/openyam/teleop_ik.py should be exercised in an environment with Git LFS, the OpenYAM model archive, and turbojpeg installed.

T-Rex T-Rex Logs

What T-Rex did

  • The Pinocchio runtime probe was executed with a deliberately reordered URDF, reduced to commanded joints, and tested against the joint mapping and velocity limits; the probe exited successfully and the contracts behaved as intended.
  • Canonical OpenYAM asset loading and Quest posture checks were attempted but blocked due to missing dependencies (git-lfs and turbojpeg), so those checks could not run.
  • The validation run documented that no failures were observed in the exercised contracts, and the logs recorded the exact commands and exit codes at the top of each captured artifact to enable review.

View all artifacts

T-Rex Ran code and verified through T-Rex

Reviews (1): Last reviewed commit: "fix(openyam): align teleop with canonica..." | Re-trigger Greptile

@TomCC7
TomCC7 force-pushed the fix/openyam-learning-baseline branch 4 times, most recently from 9421c5f to e0cac4d Compare September 1, 2026 17:17
@TomCC7
TomCC7 force-pushed the fix/openyam-learning-baseline branch from e0cac4d to 9e586e8 Compare September 1, 2026 18:54
@TomCC7
TomCC7 force-pushed the fix/openyam-learning-baseline branch from 9e586e8 to b9b719f Compare September 4, 2026 03:24
@TomCC7
TomCC7 force-pushed the fix/openyam-learning-baseline branch from b9b719f to 82dc5dc Compare September 4, 2026 19:33
@TomCC7
TomCC7 added this pull request to stack #4160 September 15, 2026 04:24
@TomCC7 TomCC7 added the backport:skip Skip creating a backport to any release branches label Sep 15, 2026
Base automatically changed from fix/webcam-linux-device-backend to main September 16, 2026 20:24
@TomCC7
TomCC7 force-pushed the fix/openyam-learning-baseline branch from 20b2bbc to bc58db8 Compare September 16, 2026 20:24
mustafab0
mustafab0 previously approved these changes Sep 17, 2026
Comment thread dimos/robot/manipulators/openyam/joints.py
mustafab0
mustafab0 previously approved these changes Sep 18, 2026
@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Sep 18, 2026
@mustafab0
mustafab0 added this pull request to the merge queue Sep 18, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to no response for status checks Sep 18, 2026
@mustafab0
mustafab0 added this pull request to the merge queue Sep 18, 2026
Merged via the queue into main with commit 26665a9 Sep 18, 2026
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip Skip creating a backport to any release branches ready-to-merge Required CI checks have passed on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants