feat(imitation): add ABC backend and shared policy rollout module

[TomCC7]

Adds ABC-DiT and LeRobot backends behind one policy module, selectable through ordinary module configuration (dimos run ... --policy.backend abc). Dual-YAM rollout blueprints connect two wrist D405s and overhead RGB, with optional Quest controls.

Stacked on #3942 (cc/feat/flexible-policy-module). This PR owns the shared policy runtime, ABC backend, and model-asset downloader used by the child WebXR audio PR #4156.

Changes

• Share observation freshness checks, motion-free inference preflight, joint-order mapping, trajectory execution, and explicit rollout start/stop across isolated backend environments.
• Use pinned upstream ABC inference code and released bottles checkpoint defaults: predict 30 actions, execute 15, then infer again.
• Add configurable typed camera ports and dual-YAM rollout blueprints with and without Quest; retain Quest A and explicit rollout start/stop.
• Accept local paths or HTTPS URLs for ABC checkpoints and normalization statistics. Preflight automatically downloads remote assets before model construction and reports failures without starting motion.
• Introduce download_http_asset() and download_hf_asset() in dimos.utils.assets. HTTPS files and HF model blobs live under CACHE_DIR/assets; existing STATE_DIR cache-use markers protect downloads and model loading from cleanup.
• Route CLIP weights and its pinned tokenizer through the HTTPS helper. Remove the upstream standalone downloader and clip_cache_dir setting; record the vendored loading adaptation without changing numerical operations.
• Use atomic HTTPS publication and per-source locking, optional SHA-256 checks, explicit HF revisions, and offline cache hits. External local checkpoints remain outside cache ownership.
• Remove obsolete LeRobot-specific module paths and document backend switching, automatic asset setup, and hardware bringup.

Validation

• 43 shared asset/cache/policy tests and 10 ABC backend tests passed, including concurrency, cleanup exclusion, failed-download retry, URL preservation, normalization precedence, and comparison with the upstream sampler.
• Targeted production and ABC backend mypy checks passed; commit hooks passed.
• Downloaded the official ABC normalization statistics and CLIP weights/tokenizer into an isolated dimOS cache. Verified the CLIP weight checksum, blocked outbound connections, reloaded from cache, and produced a 512-dimensional text embedding.
• Earlier validation of the released bottles checkpoint on an RTX 3090 produced finite 30 x 14 chunks using compiled BF16/CUDA graphs, approximately 75 ms subsequent inference, and 8.43 GB peak allocated GPU memory. That full-checkpoint GPU test was not repeated for the downloader change.

Manual validation

Use the HTTPS launch example and acceptance steps in ABC setup. Preflight downloads and validates without starting rollout. Repeat with cached assets and outbound internet blocked; confirm download failures leave the policy unready and cache cleanup refuses an active loader.

See shared model-asset downloads for cache ownership and helper usage.

Physical dual-YAM rollout, camera orientation, gripper calibration, stop/restart behavior on hardware, and task outcomes remain pending. No robot motion was performed for these changes.

Agent: Codex

[codecov]

❌ 1 Tests Failed:

Tests completed	Failed	Passed	Skipped
5755	1	5754	214

View the top 1 failed test(s) by shortest run time

dimos.codebase_checks.test_blueprint_kwargs::test_blueprint_atom_kwargs_match_module_config[unitree-g1-teleop]

Stack Traces | 0.007s run time

blueprint_name = 'unitree-g1-teleop'
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0xfff87bf7f800>

    @pytest.mark.parametrize("blueprint_name", _blueprint_params())
    def test_blueprint_atom_kwargs_match_module_config(
        blueprint_name: str, monkeypatch: pytest.MonkeyPatch
    ) -> None:
        """Fail when blueprint kwargs cannot be consumed by their target module."""
        # The multi-robot blueprints read ROBOT_IPS at import time.
        monkeypatch.setattr(global_config, "robot_ips", "192.0.2.10,192.0.2.11")
        blueprint = _get_blueprint_or_skip(blueprint_name)
    
        violations: list[str] = []
        for atom in blueprint.blueprints:
            unknown_kwargs = sorted(set(atom.kwargs) - _allowed_kwarg_names(atom.module))
            if unknown_kwargs:
                violations.append(
                    f"{atom.module.__module__}.{atom.module.__name__}: unknown kwargs {unknown_kwargs}"
                )
    
        if violations:
            listing = "\n".join(f"  - {violation}" for violation in violations)
>           raise AssertionError(
                f"Blueprint {blueprint_name!r} passes unknown module kwargs:\n{listing}\n\n"
                "Blueprint kwargs are forwarded into the module constructor. For modules "
                "with an explicit `config` annotation, use fields from that config model; "
                "for legacy modules with direct constructor parameters, use the declared "
                "`__init__` keyword names."
            )
E           AssertionError: Blueprint 'unitree-g1-teleop' passes unknown module kwargs:
E             - dimos.robot.unitree.g1.blueprints.basic.unitree_g1_teleop.G1CollectionRecorder: unknown kwargs ['db_path', 'poseless_streams']
E           
E           Blueprint kwargs are forwarded into the module constructor. For modules with an explicit `config` annotation, use fields from that config model; for legacy modules with direct constructor parameters, use the declared `__init__` keyword names.

atom       = BlueprintAtom(kwargs={'db_path': '.../dimos/recordings/session_g1_20260918_190335.db', ... type=<class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>, direction='in')), module_refs=(), instance_name=None)
blueprint  = <[RuntimeError("Failed to pull LFS file .../dimos/data/.lfs/groot.tar.gz after 3 attempts: Command.../groot.tar.gz', '--exclude=']' returned non-zero exit status 1.") raised in repr()] Blueprint object at 0xfff881c89430>
blueprint_name = 'unitree-g1-teleop'
listing    = "  - dimos.robot.unitree.g1.blueprints.basic.unitree_g1_teleop.G1CollectionRecorder: unknown kwargs ['db_path', 'poseless_streams']"
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0xfff87bf7f800>
unknown_kwargs = ['db_path', 'poseless_streams']
violations = ["dimos.robot.unitree.g1.blueprints.basic.unitree_g1_teleop.G1CollectionRecorder: unknown kwargs ['db_path', 'poseless_streams']"]

dimos/codebase_checks/test_blueprint_kwargs.py:108: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}