feat(collection): add optional speech for WebXR and desktop controls

[TomCC7]

Contribution path

• Requested collection feedback feature; no separate tracking issue.
• Stacked on feat(imitation): add ABC backend and shared policy rollout module #4088, which provides the shared asset downloader.

Problem

• Operators need audible confirmation when an episode starts, saves, or is discarded, both in WebXR and during desktop teach collection.

Solution

• Add optional speech, off by default: --tts.enabled=true on WebXR blueprints or imitation collect --tts on the desktop.
• Share collection wording and transition detection between both interfaces. Prepare three WAVs with Kokoro on CPU, then release the model.
• Download missing model assets from Hugging Face and let spaCy install its missing tokenizer. Subsequent prepared launches work offline.
• Play confirmed transitions once. Attachment and unchanged status stay silent; newer audio interrupts older audio, and disconnect stops playback.

flowchart TD
    Input[Controller buttons or terminal commands] --> Monitor[Episode monitor]
    Monitor -->|Status stream| XR[WebXR: select cached prompt]
    Monitor -->|RPC results and polling| CLI[Collection CLI: select cached prompt]
    HF[Hugging Face assets] --> Kokoro[Kokoro: prepare WAVs before controls]
    Kokoro -.->|Prepared audio| XR
    Kokoro -.->|Prepared audio| CLI
    XR -->|Existing WebSocket| Browser[Headset browser: Web Audio]
    CLI -->|sounddevice| Speakers[Desktop speakers]

Loading

How to Test

Install optional dependencies:

uv sync --extra manipulation --extra tts

• Headset: add --tts.enabled=true to your existing WebXR collection launch, then select Connect in the headset browser.
• Desktop: start a collection stack, including teach collection, then attach:

uv run --no-sync dimos imitation collect --tts

• Press Space, Space, Space, D; expect start, save, start, cancel prompts. Quest uses B to start/save and Y to discard.
• Check silent attachment/polling, interruption by newer prompts, and stopped audio on disconnect.
• Use --no-sync after setup: exact sync can remove the lazily installed tokenizer.
• Passed: 50 Python tests, five JavaScript tests, mypy and commit hooks; offline Kokoro/Chromium playback and desktop CLI smoke checks.
• Physical headset/speaker checks remain pending; desktop speaker output was mocked.

AI assistance

• Codex (GPT-6): implementation, tests, documentation, and PR description.

Checklist

• I have read and approved the CLA.

Agent: Codex

[codecov]

❌ 1 Tests Failed:

Tests completed	Failed	Passed	Skipped
5770	1	5769	214

View the top 1 failed test(s) by shortest run time

dimos.codebase_checks.test_blueprint_kwargs::test_blueprint_atom_kwargs_match_module_config[unitree-g1-teleop]

Stack Traces | 0.006s run time

blueprint_name = 'unitree-g1-teleop'
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0xffcca24d6c30>

    @pytest.mark.parametrize("blueprint_name", _blueprint_params())
    def test_blueprint_atom_kwargs_match_module_config(
        blueprint_name: str, monkeypatch: pytest.MonkeyPatch
    ) -> None:
        """Fail when blueprint kwargs cannot be consumed by their target module."""
        # The multi-robot blueprints read ROBOT_IPS at import time.
        monkeypatch.setattr(global_config, "robot_ips", "192.0.2.10,192.0.2.11")
        blueprint = _get_blueprint_or_skip(blueprint_name)
    
        violations: list[str] = []
        for atom in blueprint.blueprints:
            unknown_kwargs = sorted(set(atom.kwargs) - _allowed_kwarg_names(atom.module))
            if unknown_kwargs:
                violations.append(
                    f"{atom.module.__module__}.{atom.module.__name__}: unknown kwargs {unknown_kwargs}"
                )
    
        if violations:
            listing = "\n".join(f"  - {violation}" for violation in violations)
>           raise AssertionError(
                f"Blueprint {blueprint_name!r} passes unknown module kwargs:\n{listing}\n\n"
                "Blueprint kwargs are forwarded into the module constructor. For modules "
                "with an explicit `config` annotation, use fields from that config model; "
                "for legacy modules with direct constructor parameters, use the declared "
                "`__init__` keyword names."
            )
E           AssertionError: Blueprint 'unitree-g1-teleop' passes unknown module kwargs:
E             - dimos.robot.unitree.g1.blueprints.basic.unitree_g1_teleop.G1CollectionRecorder: unknown kwargs ['db_path', 'poseless_streams']
E           
E           Blueprint kwargs are forwarded into the module constructor. For modules with an explicit `config` annotation, use fields from that config model; for legacy modules with direct constructor parameters, use the declared `__init__` keyword names.

atom       = BlueprintAtom(kwargs={'db_path': '.../dimos/recordings/session_g1_20260918_211230.db', ... type=<class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>, direction='in')), module_refs=(), instance_name=None)
blueprint  = <[RuntimeError("Failed to pull LFS file .../dimos/data/.lfs/groot.tar.gz after 3 attempts: Command.../groot.tar.gz', '--exclude=']' returned non-zero exit status 1.") raised in repr()] Blueprint object at 0xffcca28dda00>
blueprint_name = 'unitree-g1-teleop'
listing    = "  - dimos.robot.unitree.g1.blueprints.basic.unitree_g1_teleop.G1CollectionRecorder: unknown kwargs ['db_path', 'poseless_streams']"
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0xffcca24d6c30>
unknown_kwargs = ['db_path', 'poseless_streams']
violations = ["dimos.robot.unitree.g1.blueprints.basic.unitree_g1_teleop.G1CollectionRecorder: unknown kwargs ['db_path', 'poseless_streams']"]

dimos/codebase_checks/test_blueprint_kwargs.py:108: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}