Claude
Cursor
Skill
isaac-lab
Train, imitate, evaluate, and deploy robot policies with NVIDIA Isaac Lab.
Virus-scanned
Reviewed automatically before listing.
Download
robium-ai-robium-skills_isaac-lab-498ea4e.zip · 7 KB
Install
skills CLI
npx skills add https://github.com/robium-ai/robium/tree/main/skills/isaac-lab
Claude Code
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install robium-ai-robium@llmmart
Git
git clone https://github.com/robium-ai/robium.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole robium-ai/robium collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Isaac Lab
Isaac Lab adds a training loop to a matched Isaac Sim runtime. Prove a shipped task end to end before creating a robot, environment, or reward.
Establish the runtime
- Inherit the current hardware and operating-system gate from Isaac Sim.
- Verify the supported Isaac Sim and Isaac Lab pairing; newest plus newest is not automatically compatible.
- Prefer NVIDIA's matched Isaac Lab image on a cloud GPU. Use a source install when the workstation and version pairing are intentionally maintained.
- List registered tasks from the installed release instead of guessing a task ID or script path from an older tutorial.
Prove the policy loop
- Choose the learning path explicitly: reinforcement learning from rewards, or imitation learning from demonstrations and generated variants.
- Run a known task headless with few environments and few iterations.
- Verify environment reset, observation/action shapes, reward terms, logging, and checkpoint creation before scaling parallel environments.
- Locate outputs using the training library's experiment name and current configuration, not an assumed task-name directory.
- Evaluate a named checkpoint through the matching play script. Export only after its observed behavior and metrics are useful.
- Add or change one reward, termination, terrain, or robot dimension at a time; a larger batch of edits hides which contract broke.
Go deeper only when needed
- For NVIDIA's prebuilt image on RunPod, read references/prebuilt-image-runpod.md after the cloud provider is chosen.
- For the measured Unitree Go2 RSL-RL workflow, rewards, checkpoints, and custom task route, read references/go2-rl-workflow.md.
- For teleoperation, Mimic/robomimic imitation learning, export, sim-to-sim, or hardware deployment, read IMITATION-AND-DEPLOYMENT.md.
- For runtime, output, task-registry, or interactive-viewer symptoms, start with FAILURES.md.
- Use the current Isaac Lab documentation and source for task IDs, script paths, configuration, and export behavior.
- Isaac Sim owns the underlying scene and sensors. LeRobot owns LeRobot-format dataset and real-robot training workflows; data owns the simulation-versus-real sourcing decision.
Done
- A small shipped task trains, writes a discoverable checkpoint, plays back through the matching runtime, and provides a measured baseline for any custom task or scaled run.
Files (robium)
-
references
-
go2-rl-workflow.md 4.1 KB
# Go2 locomotion: a verified RL workflow An end-to-end, battle-tested RSL-RL run on the Unitree Go2, exercised on a RunPod L4 pod with the prebuilt Isaac Lab image (3.0.0-beta2-post1), 2026-07-26..28. Facts marked "verified against 3.0.0 source" were read out of the Isaac Lab 3.0.0 source/docs in that session, not recalled. ## The verified pipeline (✓) - **Task IDs:** `Isaac-Velocity-Flat-Unitree-Go2-v0` (train), plus `Isaac-Velocity-Flat-Unitree-Go2-Play-v0` and a `Rough` variant. Confirm the current names with the installed `list_envs.py` rather than assuming these across releases. - **Train script:** `scripts/reinforcement_learning/rsl_rl/train.py`. Default RL library is `rsl_rl` (PPO). - **Smoke run:** `--num_envs 32 --max_iterations 10` → exit code 0, checkpoints written as `model_<iter>.pt`. - **Benign headless warnings** to ignore on a clean run: OmniHub, X Server, materialx, usdrt. ## Critical: checkpoints land under the EXPERIMENT NAME, not the task ID Checkpoints go to `logs/rsl_rl/unitree_go2_flat/<timestamp>/`: the subdir is the **experiment name** `unitree_go2_flat`, **not** the task ID `Isaac-Velocity-Flat-Unitree-Go2-v0`. A smoke test that asserts on a task-ID-shaped log path will fail even though training succeeded. Assert on the experiment-name path. ## Full training profile - **4096 envs converges by ~iteration 300** (reward climbs from ≈ −0.5 to a ≈ +36 plateau; episode length saturates at the 1000-step max). The 1000–2000-iteration defaults are refinement, not required for a walking policy; stop early once the plateau holds. - **`save_interval` is a Hydra override**, not a CLI flag: `agent.save_interval=N` on the train command. - **Render videos post-hoc**, not during training: run `play.py --checkpoint <path> --video --enable_cameras` (~2 min per checkpoint). This is preferred over in-training `--video`, which slows the run for footage you usually only want from a few checkpoints. ## Reward and cost live in CONFIG, not code Reward and cost share a single `RewardsCfg`: a **cost is just a term with a negative weight**. The per-step reward is `Σ weight · term() · dt`. Relevant files (verified against 3.0.0 source): - `locomotion/velocity/velocity_env_cfg.py`: the base `RewardsCfg`. - `locomotion/velocity/mdp/rewards.py`: task-specific reward term functions. - `isaaclab/envs/mdp/rewards.py`: the generic reward term library. Weights override down a **3-layer chain**: `velocity_env_cfg` → `config/go2/rough_env_cfg` → `config/go2/flat_env_cfg`, with the flat config winning for the flat task. Kernels: reward terms use `exp(-err²/std²)`; penalty terms use `sum(square(x))`. ## Custom robot / new task = EXTERNAL project, not a fork To add a robot or task, scaffold an **isolated external project** with `./isaaclab.sh --new`. It creates a standalone repo that pip-installs Isaac Lab as a dependency and `gym.register`s the new task; you do not fork or edit the Isaac Lab repo. The "internal task" path (editing the Isaac Lab repo in place) is only for upstreaming, and it is **auto-disabled when Isaac Lab is pip-installed**, which is exactly the case inside the prebuilt NGC container. So on the prebuilt image, external-project is the only path. (Verified vs the 3.0.0 doc: overview / own-project / template.rst.) ## Checkpoint portability across patch releases (✓) An RSL-RL checkpoint (`model_1999.pt`, 984 KB) trained on Isaac Lab 3.0.0-beta2 loaded cleanly into 3.0.0-beta2-post1: same task → identical observation/action dims and MLP shape, no size mismatch. This skips a 20–30-minute retrain when moving between patch images. ## Running custom scripts on a fresh pod - **`/workspace` can be read-only for the `ubuntu` user** on a fresh pod → write your scripts and logs to `/data` (the non-`/workspace` volume path from the prebuilt-image-runpod reference) instead. - **`play.py` imports a sibling `cli_args` module**, so launching it from `/data` throws `ModuleNotFoundError: cli_args`. Fix: set `PYTHONPATH` to the rsl_rl scripts directory (where `cli_args.py` lives) before running a copied/relocated `play.py`. -
prebuilt-image-runpod.md 3 KB
# Prebuilt Isaac Lab image + RunPod provisioning The battle-tested way to get Isaac Lab onto a cloud GPU without the version-pairing trap or a source install: pull NVIDIA's prebuilt Isaac Lab container, which bundles a matched Isaac Sim + Isaac Lab in one image. Observed on a RunPod L4 pod running the go2-locomotion app, 2026-07-26..28. ## Prefer the prebuilt image over source install - **Image:** `nvcr.io/nvidia/isaac-lab`, latest tag observed **3.0.0-beta2-post1** (observed on the NGC catalog 2026-07-26..28). It ships a matched Isaac Sim + Isaac Lab together, so it sidesteps the Isaac-Sim/Isaac-Lab version-pairing trap (see the skill's Platform gotchas) and the multi-step source install entirely: no `pip install isaacsim` + `./isaaclab.sh --install` on top. - **Observed pod environment** (state as observed, not as a floor; re-verify per pod): NVIDIA driver **580.159.04**, Python **3.12.13**. Note the driver is what the pod host happened to ship; it is not the Isaac Sim standalone minimum the sibling `isaac-sim` skill records. - The image is Isaac Lab pip-installed into the container, which has a downstream consequence for custom tasks; see the external-project note in the go2-rl-workflow reference. ## Provisioning specifics (RunPod) - **NGC pull auth.** The image lives on NGC, so the pod needs registry credentials: supply them through RunPod's `containerRegistryAuth` (an NGC API key), not a bare public pull. - **EULA / privacy env vars.** Set `ACCEPT_EULA=Y` and `PRIVACY_CONSENT=Y` in the pod's environment or the container refuses to start, the same gate the `isaac-sim` container uses. - **Volume-shadow gotcha (the one that hides Isaac Lab).** RunPod's default persistent volume mounts at `/workspace`. The prebuilt image places Isaac Lab under `/workspace` too, so the volume mount *shadows* the bundled install: Isaac Lab appears to vanish. Fix: set the pod's `volumeMountPath` to a non-`/workspace` path (e.g. `/data`) so the persistent volume and the bundled Isaac Lab don't collide. - **Custom-image entrypoint (dockerStartCmd becomes ARGS).** The image defines its own `ENTRYPOINT`, which means RunPod's `dockerStartCmd` is passed to that entrypoint as *arguments*, not run as a shell command, so a shell one-liner in `dockerStartCmd` silently does the wrong thing. Override `dockerEntrypoint` to `["/bin/bash","-lc"]` so the start command runs as a shell. After changing entrypoint/args, **stop then start** the pod; RunPod's `/restart` returns HTTP 500 for this change; a clean stop→start applies it. ## General Pod operations belong to the runpod skill Pod port exposure, the dead RunPod proxy, SSH-exec limits, datacenter enumeration, and `runpodctl` are general RunPod/GPU-cloud mechanics owned by the `runpod` skill; verify its current provider sources instead of re-deriving them here. This reference covers only what is Isaac-Lab-image-specific on top of that groundwork (the auth, EULA, volume-shadow, and entrypoint items above).
-
-
FAILURES.md 2.3 KB
# When Isaac Lab fails Start below the policy: runtime, task registration, environment, training, then artifact. ## Isaac Lab does not import or start - Verify the Isaac Sim/Lab compatibility pair and how both were installed. - On cloud hosts, prefer a matched prebuilt image before layering source installs into an unknown runtime. - Confirm the Isaac Sim hardware and driver gate independently. ## The task cannot be found - List the installed registry and compare the exact task entry point. Task IDs and script directories move across releases. - In a pip-installed/prebuilt environment, make a custom task an external project that registers itself. Robium observed the internal-task route being disabled there because it is meant for upstream development. ## Training exits cleanly but the checkpoint seems missing - Inspect the training library's experiment name and resolved configuration. Robium's Go2 RSL-RL run wrote under `unitree_go2_flat`, not the full task ID. - Confirm save interval and output mount before rerunning paid compute. - On the prebuilt image, a persistent volume mounted at `/workspace` can hide the bundled application. Robium used a non-overlapping path such as `/data`. ## Training is slow or unstable - Reduce environments and iterations until resets and metrics are observable. - Separate data loading, physics, rendering, and policy-update throughput. - Run full training headless. Produce selected videos afterward rather than paying the rendering cost throughout training. - Inspect the resolved reward configuration. In the verified Go2 task, costs were negative-weight reward terms and robot-specific configs overrode base weights through multiple layers. ## Playback or custom scripts fail - Use the play script from the same library and release as the training run. - A copied RSL-RL play script may import a sibling `cli_args` module; keep the script tree intact or provide the corresponding module path. - Verify observation/action dimensions before assuming a checkpoint is portable across versions. Robium observed portability across one patch pair only. - Interactive editor control and policy stepping can compete for the simulation timeline. Use headless execution and a view-only stream unless the combination is proven on the target release. -
IMITATION-AND-DEPLOYMENT.md 2.8 KB
# Imitation learning and deployment Use this card only when the task goes beyond the common RL train/play loop. Isaac Lab's Mimic, teleoperation, robomimic scripts, exporters, and deployment examples change quickly; the current official workflow is the source of truth. ## Imitation learning - Start from an installed task that supports the selected teleoperation device and demonstration schema. Prove reset, task completion, and one replay before collecting volume. - Keep the stages distinct: collect demonstrations, annotate or inspect success, generate augmented demonstrations with Mimic when appropriate, train through the supported imitation library, then evaluate named checkpoints. - Generated demonstrations inherit the source task, scene, controller, and success predicate. Inspect their coverage and failures rather than treating a successful generation command as valid training data. - The current official examples use Isaac Lab Mimic and robomimic, but script names, extras, tasks, and dataset formats are release-sensitive. Follow the [teleoperation and imitation guide](https://isaac-sim.github.io/IsaacLab/main/source/overview/imitation-learning/teleop_imitation.html) for the installed release. - Cross into `data` for source-mix and demonstration-quality decisions. Cross into `lerobot` only when the artifact is a LeRobotDataset or the policy is trained through LeRobot. ## Export and deployment - Playback is not deployment. First record the exact observation ordering, normalization, action scaling, recurrent state, control rate, and actuator limits that wrap the checkpoint. - Prefer an exporter that preserves those semantics. Current Isaac Lab documentation includes LEAPP for supported manager-based RL workflows and policy-deployment examples; confirm the learning library, environment type, physics preset, and optional extras before choosing it. - Re-run the policy in a second simulation or deployment harness before hardware when that can expose hidden simulator dependencies. - Hardware rollout adds timing, estimator, actuator, network, and safety contracts. Begin with bounded commands and an independent stop path; do not infer safe behavior from the Isaac Lab playback video. - Use the current [policy deployment guide](https://isaac-sim.github.io/IsaacLab/main/source/policy_deployment/index.html) and the selected robot/example repository for the actual export and hardware bridge. Do not generalize a HOVER, gear-insertion, ROS, or LEAPP recipe to an unrelated task. ## Evidence to keep - Matched Isaac Lab/Isaac Sim release and physics preset. - Dataset or reward configuration and task revision. - Checkpoint plus preprocessing, postprocessing, and export metadata. - Sim playback, cross-simulator result when used, control frequency, and the exact hardware safety envelope. -
SKILL.md 2.7 KB
--- name: isaac-lab description: Train, imitate, evaluate, and deploy robot policies with NVIDIA Isaac Lab. --- # Isaac Lab Isaac Lab adds a training loop to a matched Isaac Sim runtime. Prove a shipped task end to end before creating a robot, environment, or reward. ## Establish the runtime - Inherit the current hardware and operating-system gate from Isaac Sim. - Verify the supported Isaac Sim and Isaac Lab pairing; newest plus newest is not automatically compatible. - Prefer NVIDIA's matched Isaac Lab image on a cloud GPU. Use a source install when the workstation and version pairing are intentionally maintained. - List registered tasks from the installed release instead of guessing a task ID or script path from an older tutorial. ## Prove the policy loop - Choose the learning path explicitly: reinforcement learning from rewards, or imitation learning from demonstrations and generated variants. - Run a known task headless with few environments and few iterations. - Verify environment reset, observation/action shapes, reward terms, logging, and checkpoint creation before scaling parallel environments. - Locate outputs using the training library's experiment name and current configuration, not an assumed task-name directory. - Evaluate a named checkpoint through the matching play script. Export only after its observed behavior and metrics are useful. - Add or change one reward, termination, terrain, or robot dimension at a time; a larger batch of edits hides which contract broke. ## Go deeper only when needed - For NVIDIA's prebuilt image on RunPod, read [references/prebuilt-image-runpod.md](references/prebuilt-image-runpod.md) after the cloud provider is chosen. - For the measured Unitree Go2 RSL-RL workflow, rewards, checkpoints, and custom task route, read [references/go2-rl-workflow.md](references/go2-rl-workflow.md). - For teleoperation, Mimic/robomimic imitation learning, export, sim-to-sim, or hardware deployment, read [IMITATION-AND-DEPLOYMENT.md](IMITATION-AND-DEPLOYMENT.md). - For runtime, output, task-registry, or interactive-viewer symptoms, start with [FAILURES.md](FAILURES.md). - Use the current [Isaac Lab documentation](https://isaac-sim.github.io/IsaacLab/) and [source](https://github.com/isaac-sim/IsaacLab) for task IDs, script paths, configuration, and export behavior. - Isaac Sim owns the underlying scene and sensors. LeRobot owns LeRobot-format dataset and real-robot training workflows; data owns the simulation-versus-real sourcing decision. ## Done - A small shipped task trains, writes a discoverable checkpoint, plays back through the matching runtime, and provides a measured baseline for any custom task or scaled run.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.