ioailab

ioailab provides G1 robot cfgs, an IsaacLab-style task registry, action/sensor helpers, dataset refs, and action agents for IsaacLab.

Quick Start

Build once and enter the GUI container (also the GP001 teleop shell):

make build
make shell-gui

make_env(...) handles app launch, task registration, and env construction in one call. Agents return full action tensors, and env.collect(...) records data through IsaacLab's recorder manager:

from ioailab.agents import CuroboPlannerAgent
from ioailab.envs import make_env

task_id = "GalbotG1-PickCube-v0"
env = make_env(task_id, num_envs=1)
agent = CuroboPlannerAgent.from_task(task_id)

dataset = env.collect(
    agent=agent,
    episodes=1,
    path="data/pick_cube_demos.hdf5",
)
env.close()

env.collect(...) exports data only when the env reports terminated/truncated or when the caller's max_steps limit is reached. env.is_running() is only the Isaac app-lifecycle guard.

The data pipeline (collect → Mimic → train → evaluate) runs as separate processes so IsaacSim state never leaks across stages:

python examples/01_collect.py   # collect motion-planner data; teleop is shown in comments
python examples/02_mimic.py     # mimic(dataset, episodes=...) expansion
python examples/03_train.py --export-inference  # robomimic_diffusion training + export
python examples/04_eval.py --checkpoint <run>/inference/model_inference.pth --headless

Use examples/06_collect_component_task.py for PickToShelf/SortToShelf component-task data, and examples/07_compound_task.py for their coherent task runs. Use examples/policy_baseline/01_collect_ioai_pick.py --product <id> for strict, product-specific IOAI Pick datasets. Use examples/policy_baseline/03_collect_ioai_sim_scene.py --product <id> to run the continuous IOAI Pick→Nav→Place pipeline, optionally retain its HDF5 dataset, and save either one Nav→Place Scenario or a numbered Scenario collection. Add --pick-checkpoint <path> to replace only the default planner-driven Pick phase with a trained Pick policy. Pass the Scenario YAML or directory to examples/policy_baseline/02_collect_ioai_place.py --init-scenario ... to collect standalone Place data from real Pick/Nav terminal states instead of a hard-coded grasp. Add --no-rgb to the standalone Place collector to omit front_head_rgb from the HDF5 while retaining actions, robot joint observations, and simulator states for low-dimensional policy training. Evaluate a standalone Place checkpoint with examples/04_eval.py --product <id>; it restores data/ioai_sim_scene_place/scenarios/<product>.yaml by default. Override that path with --init-scenario, using a {product} placeholder when evaluating --product all. Each standalone reset samples one Scenario from the collection, then perturbs only the selected product in the active gripper frame: local x/y use +/-2 mm and local z uses -3 mm to +1 mm. Product orientation and gripper joints stay unchanged. The nine non-cocoa products are supported; cocoa remains an independent Pick-only black-tray transfer.

Use examples/generate_ioai_table_layout.py to save one of the 12 supported table arrangements, then run examples/policy_baseline/04_eval_ioai_policy_pipeline.py with that layout and outputs/ioai_policy_baseline.yaml. The baseline routes each known product to its exact Pick/Place policies through TaskFlowAgent, keeps Nav task-owned, and lazily retains at most the configured number of models. For motion-planning examples, use cuRobo v2 (curobov2). Registered task IDs

Traditional Vision Baseline (Local GPU Only)

Warning

examples/vision_baseline/ cannot run on the provided remote machine because of GPU limits. The baseline is verified on local NVIDIA GPU machines; clone the repository and run it there.

YOLO setup · FoundationPose pick baseline

📖 Documentation

Detailed documentation can be found at:

Online Documentation

Topic	Page
Tutorial	docs/tutorial.md
Examples	docs/examples.md
Architecture	docs/architecture.md
Action agents & task flows	docs/agents.md
Tasks	docs/tasks.md
Data & datasets (Mimic, LeRobot v3)	docs/data.md
Sensors and cameras	docs/galbot_sensors.md
Robot reference (joints, assets)	docs/reference.md
Development workflow	docs/development.md

Architecture-sensitive work should follow AGENTS.md and docs/architecture.md.