feat(teleop): add PICO WebXR body tracking

[TomCC7]

Contribution path

• Stacked on chore(teleop): rename Quest stack to WebXR #3676

Problem

The renamed WebXR teleop stack has no way to request PICO body-tracking permission, decode the browser's joint poses, or inspect tracking health on hardware.

Solution

• Add a device-neutral BodyTrackingSnapshot output and strict JSON decoder.
• Keep body tracking off by default, with explicit optional and required WebXR session modes.
• Capture XRFrame.body poses in bounded-floor space when available and fall back to local-floor.
• Add a PICO API-test blueprint and live monitor without coupling body data to controller or viewer poses.
• Accept PICO's six-button WebXR controller packets, where the platform-reserved menu button is omitted.

How to Test

Run the hardware-facing API test:

dimos run demo-pico-body-tracking

Automated checks run locally:

CI=1 uv run pytest dimos/teleop/webxr dimos/robot/test_all_blueprints_generation.py dimos/robot/test_all_blueprints.py -q
uv run ruff check dimos/teleop/webxr dimos/robot/all_blueprints.py
uv run ruff format --check dimos/teleop/webxr dimos/robot/all_blueprints.py
uv run --group lint mypy --disable-error-code=misc dimos/teleop/webxr
node --check dimos/teleop/webxr/web/static/teleop.js
node --check dimos/teleop/webxr/web/static/webxr_body.mjs

Result: 150 passed, 3 skipped, 40 deselected.

Physical PICO validation remains to be completed with the command above.

AI assistance

OpenAI Codex with GPT-5 implemented the change, resolved the parent rebase conflicts, and ran the checks. The author should review and understand the final diff before merge.

Checklist

• I have read and approved the CLA.

[codecov]

Codecov Report

❌ Patch coverage is 99.65753% with 1 line in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
dimos/teleop/webxr/module.py	98.59%	0 Missing and 1 partial ⚠️

@@            Coverage Diff             @@
##             main    #3695      +/-   ##
==========================================
+ Coverage   78.89%   78.96%   +0.06%     
==========================================
  Files        1469     1471       +2     
  Lines      138999   139279     +280     
  Branches    11941    12172     +231     
==========================================
+ Hits       109665   109975     +310     
+ Misses      25934    25903      -31     
- Partials     3400     3401       +1     

Components	Coverage Δ
Tests	94.77% <100.00%> (+0.01%)	⬆️

Flag	Coverage Δ
OS-ubuntu-24.04-arm	74.93% <99.65%> (+0.07%)	⬆️
OS-ubuntu-latest	75.63% <99.65%> (+0.07%)	⬆️
Py-3.10	75.40% <99.65%> (+0.07%)	⬆️
Py-3.11	75.61% <99.65%> (+0.07%)	⬆️
Py-3.12	75.61% <99.65%> (+0.07%)	⬆️
SelfHosted-Large	30.72% <23.97%> (-0.02%)	⬇️
SelfHosted-Linux	36.11% <23.97%> (-0.03%)	⬇️
SelfHosted-macOS	35.44% <23.97%> (-0.03%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
dimos/robot/all_blueprints.py	100.00% <ø> (ø)
dimos/teleop/webxr/blueprints.py	100.00% <100.00%> (ø)
dimos/teleop/webxr/body_tracking.py	100.00% <100.00%> (ø)
dimos/teleop/webxr/controller_types.py	96.55% <100.00%> (+4.59%)	⬆️
dimos/teleop/webxr/test_blueprints.py	100.00% <100.00%> (ø)
dimos/teleop/webxr/test_body_tracking.py	100.00% <100.00%> (ø)
dimos/teleop/webxr/test_module.py	100.00% <100.00%> (ø)
dimos/utils/logging_config.py	81.20% <100.00%> (+1.34%)	⬆️
dimos/utils/test_logging_config.py	100.00% <100.00%> (ø)
dimos/teleop/webxr/module.py	81.03% <98.59%> (+10.01%)	⬆️

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[greptile-apps]

Confidence Score: 5/5

No blocking failure remains; the browser loaded and executed the updated WebXR client assets.

What we checked:

• A headless browser session loaded /teleop and the teleop assets loaded successfully, with /static/teleop.js and /static/webxr_body.mjs returning 200 and window.connect defined after module evaluation.
• A controlled import-change test substituted the relative import with a missing module, which yielded GET /static/missing_webxr_body.mjs as 404 and window.connect undefined, confirming the current successful load depends on the existing static dependency.
• A review of teleop.js:8 corroborated that there is no vulnerability and the suggested P1 regression is disproved.

Summary

• This update adds configurable PICO WebXR body-tracking capture, heartbeat reporting, monitoring, and shared-memory attachment hardening. A browser check confirmed that the teleoperation page loads /static/webxr_body.mjs successfully and that the client module executes. No blocking failure remains.
• Merge safety: safe to merge.

_{Reviews (6) · Last reviewed commit: "fix(shm): reject zero-sized attachment h..."}

[TomCC7]

Addressed the Greptile P1 in 82d7bc6. WebXRControllerState now requires the six button slots it indexes while keeping the seventh menu button optional. A regression test sends a real five-button LCM Joy packet through WebXRTeleopModule and verifies the packet is rejected without escaping the callback, stale controller state is cleared, and zero velocity is published. Local verification: 54 WebXR tests passed; Ruff and mypy passed.

[TomCC7]

@greptileai please re-review the latest commit 82d7bc6. The prior P1 about short controller packets has been fixed and regression-tested.

[TomCC7]

@greptileai please re-review the latest commit 23e047e. The branch now includes current main, explicit body-tracking heartbeats, and the SHM race fix.

— Codex (AI agent)

[github-actions]

Successfully created backport PR for release/0.0.14:

• [Backport release/0.0.14] feat(teleop): add PICO WebXR body tracking #4178