Skip to content

K
KincoActuatorIsaacLab
Unverified Commit 454c3d2f authored by Kelly Guo's avatar Kelly Guo Committed by GitHub
Browse files
Options

Merges release 2.3 changes into main with Isaac Sim 5.1 support (#3857) 

# Description

Merging in updates for the Isaac Lab 2.3 release, moving to support for
Isaac Sim 5.1.
This change includes many features for teleoperation, disjoint
navigation, whole-body control for teleoperation, and IK updates from
@rwiltz, @michaellin6, @jaybdub, @huihuaNvidia2023, @hougantc-nvda,
@lotusl-code, @yami007007, @cathyliyuanchen, @tifchen-nvda.
Additionally, support for DGX Spark is added by @ooctipus and
@matthewtrepte.

For details of the changes and updates, refer to the release notes.

## Type of change

- New feature (non-breaking change which adds functionality)
- Breaking change (existing functionality will not work without user
modification)
- Documentation update


## Checklist

- [x] I have read and understood the [contribution
guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html)
- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

<!--
As you go through the checklist above, you can mark something as done by
putting an x character in it

For example,
- [x] I have done this task
- [ ] I have not done this task
-->

---------
Signed-off-by: 's avatarHarsh Patel <hapatel@theaiinstitute.com>
Signed-off-by: 's avatarrebeccazhang0707 <168459200+rebeccazhang0707@users.noreply.github.com>
Signed-off-by: 's avatarKelly Guo <kellyg@nvidia.com>
Signed-off-by: 's avataryami007007 <weihuaz@nvidia.com>
Signed-off-by: 's avatarKelly Guo <kellyguo123@hotmail.com>
Signed-off-by: 's avatarLouis LE LAY <le.lay.louis@gmail.com>
Signed-off-by: 's avatarMayank Mittal <12863862+Mayankm96@users.noreply.github.com>
Signed-off-by: 's avatarJavier Felix-Rendon <javierfelixrendon@gmail.com>
Signed-off-by: 's avatarDoug Fulop <dougfulop@gmail.com>
Signed-off-by: 's avatarMilad Rakhsha <mrakhsha@nvidia.com>
Signed-off-by: 's avatarGiulio Romualdi <giulio.romualdi@gmail.com>
Signed-off-by: 's avatarzehao-wang <59912787+zehao-wang@users.noreply.github.com>
Signed-off-by: 's avatarMichael Gussert <michael@gussert.com>
Signed-off-by: 's avatarooctipus <zhengyuz@nvidia.com>
Signed-off-by: 's avatarshauryadNv <shauryad@nvidia.com>
Co-authored-by: 's avatarPhilipp Reist <66367163+preist-nvidia@users.noreply.github.com>
Co-authored-by: 's avatarHarsh Patel <hapatel@theaiinstitute.com>
Co-authored-by: 's avatarJames Tigue <jtigue@theaiinstitute.com>
Co-authored-by: 's avatarJames Tigue <166445701+jtigue-bdai@users.noreply.github.com>
Co-authored-by: 's avatarooctipus <zhengyuz@nvidia.com>
Co-authored-by: 's avatarrebeccazhang0707 <168459200+rebeccazhang0707@users.noreply.github.com>
Co-authored-by: 's avatarmichaellin6 <michalin@nvidia.com>
Co-authored-by: 's avatarHuihua Zhao <huihuaz@nvidia.com>
Co-authored-by: 's avatarRafael Wiltz <rwiltz@nvidia.com>
Co-authored-by: 's avatarSergey Grizan <sgrizan@nvidia.com>
Co-authored-by: 's avatarAlexander Poddubny <143108850+nv-apoddubny@users.noreply.github.com>
Co-authored-by: 's avatarJohn <jaybdub@users.noreply.github.com>
Co-authored-by: 's avataryami007007 <weihuaz@nvidia.com>
Co-authored-by: 's avatarWeihua Zhang <weihuaz@weihuaz-mlt.client.nvidia.com>
Co-authored-by: 's avatarPeterL-NV <petliu@nvidia.com>
Co-authored-by: 's avatarMayank Mittal <12863862+Mayankm96@users.noreply.github.com>
Co-authored-by: 's avatarhougantc-nvda <127865892+hougantc-nvda@users.noreply.github.com>
Co-authored-by: 's avatarpeterd-NV <peterd@nvidia.com>
Co-authored-by: 's avatarnjawale42 <njawale@nvidia.com>
Co-authored-by: 's avatarCathy Li <40371641+cathyliyuanchen@users.noreply.github.com>
Co-authored-by: 's avatarLouis LE LAY <le.lay.louis@gmail.com>
Co-authored-by: 's avatarJavier Felix-Rendon <javierfelixrendon@gmail.com>
Co-authored-by: 's avatarDoug Fulop <dougfulop@gmail.com>
Co-authored-by: 's avatarRobin Vishen <117207232+vi7n@users.noreply.github.com>
Co-authored-by: 's avatar-T.K.- <t_k_233@outlook.com>
Co-authored-by: 's avatarMayank Mittal <mittalma@leggedrobotics.com>
Co-authored-by: 's avatarRebecca Zhang <rebeccaz@nvidia.com>
Co-authored-by: 's avatarLorenz Wellhausen <lorenwel@users.noreply.github.com>
Co-authored-by: 's avatarLorenz Wellhausen <lorenz.wellhausen@rivr.ai>
Co-authored-by: 's avatarMichael Gussert <michael@gussert.com>
Co-authored-by: 's avatarAntoine RICHARD <antoiner@nvidia.com>
Co-authored-by: 's avatartifchen-nvda <tifchen@nvidia.com>
Co-authored-by: 's avatarCathy Li <yuanchenl@yuanchenl-mlt.client.nvidia.com>
Co-authored-by: 's avatarrwiltz <165190220+rwiltz@users.noreply.github.com>
Co-authored-by: 's avatarMilad-Rakhsha-NV <167464435+Milad-Rakhsha-NV@users.noreply.github.com>
Co-authored-by: 's avatarMilad-Rakhsha <miladrakhsha@gmail.com>
Co-authored-by: 's avatarGiulio Romualdi <giulio.romualdi@gmail.com>
Co-authored-by: 's avatarXinjie Yao <xyao@nvidia.com>
Co-authored-by: 's avatarshauryadNv <shauryad@nvidia.com>
Co-authored-by: 's avatarToni-SM <aserranomuno@nvidia.com>
Co-authored-by: 's avatarzehao-wang <59912787+zehao-wang@users.noreply.github.com>
Co-authored-by: 's avataryijieg <yijieg@nvidia.com>
Co-authored-by: 's avatarndahile-nvidia <167997649+ndahile-nvidia@users.noreply.github.com>
Co-authored-by: 's avatarmatthewtrepte <mtrepte@nvidia.com>
Co-authored-by: 's avataryanziz-nvidia <yanziz@nvidia.com>
Co-authored-by: 's avatarlotusl-code <lotusl@nvidia.com>
parent f6cd8763
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 9336 additions and 773 deletions
.github/workflows/build.yml
View file @ 454c3d2f
......@@ -26,7 +26,7 @@ permissions:
env:
NGC_API_KEY: ${{ secrets.NGC_API_KEY }}
ISAACSIM_BASE_IMAGE: ${{ vars.ISAACSIM_BASE_IMAGE || 'nvcr.io/nvidia/isaac-sim' }}
ISAACSIM_BASE_VERSION: ${{ vars.ISAACSIM_BASE_VERSION || '5.0.0' }}
ISAACSIM_BASE_VERSION: ${{ vars.ISAACSIM_BASE_VERSION || '5.1.0' }}
DOCKER_IMAGE_TAG: isaac-lab-dev:${{ github.event_name == 'pull_request' && format('pr-{0}', github.event.pull_request.number) || github.ref_name }}-${{ github.sha }}
jobs:
......
.github/workflows/license-exceptions.json
View file @ 454c3d2f
......@@ -431,6 +431,11 @@
"license": "UNKNOWN",
"comment": "Apache 2.0"
},
{
"package": "onnx-ir",
"license": "UNKNOWN",
"comment": "Apache 2.0"
},
{
"package": "matplotlib-inline",
"license": "UNKNOWN",
......
.github/workflows/postmerge-ci.yml
View file @ 454c3d2f
......@@ -23,7 +23,7 @@ permissions:
env:
NGC_API_KEY: ${{ secrets.NGC_API_KEY }}
ISAACSIM_BASE_IMAGE: ${{ vars.ISAACSIM_BASE_IMAGE || 'nvcr.io/nvidia/isaac-sim' }}
ISAACSIM_BASE_VERSIONS_STRING: ${{ vars.ISAACSIM_BASE_VERSIONS_STRING || 'latest-base-5.0' }}
ISAACSIM_BASE_VERSIONS_STRING: ${{ vars.ISAACSIM_BASE_VERSIONS_STRING || '5.1.0' }}
ISAACLAB_IMAGE_NAME: ${{ vars.ISAACLAB_IMAGE_NAME || 'isaac-lab-base' }}
jobs:
......
CITATION.cff
View file @ 454c3d2f
cff-version: 1.2.0
message: "If you use this software, please cite both the Isaac Lab repository and the Orbit paper."
title: Isaac Lab
version: 2.2.1
version: 2.3.0
repository-code: https://github.com/NVIDIA-Omniverse/IsaacLab
type: software
authors:
......
CONTRIBUTORS.md
View file @ 454c3d2f
......@@ -102,6 +102,7 @@ Guidelines for modifications:
* Miguel Alonso Jr
* Mingyu Lee
* Muhong Guo
* Narendra Dahile
* Neel Anand Jawale
* Nicola Loi
* Norbert Cygiert
......@@ -123,6 +124,7 @@ Guidelines for modifications:
* Ritvik Singh
* Rosario Scalise
* Ryley McCarroll
* Sergey Grizan
* Shafeef Omar
* Shaoshu Su
* Shaurya Dewan
......@@ -130,6 +132,7 @@ Guidelines for modifications:
* Shundo Kishi
* Stefan Van de Mosselaer
* Stephan Pleines
* Tiffany Chen
* Tyler Lum
* Victor Khaustov
* Virgilio Gómez Lambo
......
README.md
View file @ 454c3d2f
......@@ -4,7 +4,7 @@
# Isaac Lab
[![IsaacSim](https://img.shields.io/badge/IsaacSim-5.0.0-silver.svg)](https://docs.isaacsim.omniverse.nvidia.com/latest/index.html)
[![IsaacSim](https://img.shields.io/badge/IsaacSim-5.1.0-silver.svg)](https://docs.isaacsim.omniverse.nvidia.com/latest/index.html)
[![Python](https://img.shields.io/badge/python-3.11-blue.svg)](https://docs.python.org/3/whatsnew/3.11.html)
[![Linux platform](https://img.shields.io/badge/platform-linux--64-orange.svg)](https://releases.ubuntu.com/22.04/)
[![Windows platform](https://img.shields.io/badge/platform-windows--64-orange.svg)](https://www.microsoft.com/en-us/)
......@@ -54,12 +54,13 @@ Isaac Lab is built on top of Isaac Sim and requires specific versions of Isaac S
release of Isaac Lab. Below, we outline the recent Isaac Lab releases and GitHub branches and their corresponding
dependency versions for Isaac Sim.
| Isaac Lab Version | Isaac Sim Version |
| ----------------------------- | ------------------- |
| `main` branch | Isaac Sim 4.5 / 5.0 |
| `v2.2.X` | Isaac Sim 4.5 / 5.0 |
| `v2.1.X` | Isaac Sim 4.5 |
| `v2.0.X` | Isaac Sim 4.5 |
| Isaac Lab Version | Isaac Sim Version |
| ----------------------------- | ------------------------- |
| `main` branch | Isaac Sim 4.5 / 5.0 |
| `v2.3.X` | Isaac Sim 4.5 / 5.0 / 5.1 |
| `v2.2.X` | Isaac Sim 4.5 / 5.0 |
| `v2.1.X` | Isaac Sim 4.5 |
| `v2.0.X` | Isaac Sim 4.5 |
## Contributing to Isaac Lab
......
apps/isaaclab.python.headless.kit
View file @ 454c3d2f
......@@ -5,7 +5,7 @@
[package]
title = "Isaac Lab Python Headless"
description = "An app for running Isaac Lab headlessly"
version = "2.2.1"
version = "2.3.0"
# That makes it browsable in UI with "experience" filter
keywords = ["experience", "app", "isaaclab", "python", "headless"]
......@@ -15,7 +15,7 @@ keywords = ["experience", "app", "isaaclab", "python", "headless"]
app.versionFile = "${exe-path}/VERSION"
app.folder = "${exe-path}/"
app.name = "Isaac-Sim"
app.version = "5.0.0"
app.version = "5.1.0"
##################################
# Omniverse related dependencies #
......@@ -108,7 +108,7 @@ metricsAssembler.changeListenerEnabled = false
###############################
[settings.exts."omni.kit.registry.nucleus"]
registries = [
{ name = "kit/default", url = "https://ovextensionsprod.blob.core.windows.net/exts/kit/prod/106/shared" },
{ name = "kit/default", url = "https://ovextensionsprod.blob.core.windows.net/exts/kit/prod/107/shared" },
{ name = "kit/sdk", url = "https://ovextensionsprod.blob.core.windows.net/exts/kit/prod/sdk/${kit_version_short}/${kit_git_hash}" },
{ name = "kit/community", url = "https://dw290v42wisod.cloudfront.net/exts/kit/community" },
]
......@@ -215,6 +215,6 @@ enabled=true # Enable this for DLSS
# set the S3 directory manually to the latest published S3
# note: this is done to ensure prior versions of Isaac Sim still use the latest assets
[settings]
persistent.isaac.asset_root.default = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0"
persistent.isaac.asset_root.cloud = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0"
persistent.isaac.asset_root.nvidia = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0"
persistent.isaac.asset_root.default = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1"
persistent.isaac.asset_root.cloud = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1"
persistent.isaac.asset_root.nvidia = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1"
apps/isaaclab.python.headless.rendering.kit
View file @ 454c3d2f
......@@ -9,7 +9,7 @@
[package]
title = "Isaac Lab Python Headless Camera"
description = "An app for running Isaac Lab headlessly with rendering enabled"
version = "2.2.1"
version = "2.3.0"
# That makes it browsable in UI with "experience" filter
keywords = ["experience", "app", "isaaclab", "python", "camera", "minimal"]
......@@ -32,7 +32,7 @@ cameras_enabled = true
app.versionFile = "${exe-path}/VERSION"
app.folder = "${exe-path}/"
app.name = "Isaac-Sim"
app.version = "5.0.0"
app.version = "5.1.0"
### FSD
app.useFabricSceneDelegate = true
......@@ -105,7 +105,7 @@ metricsAssembler.changeListenerEnabled = false
[settings.exts."omni.kit.registry.nucleus"]
registries = [
{ name = "kit/default", url = "https://ovextensionsprod.blob.core.windows.net/exts/kit/prod/106/shared" },
{ name = "kit/default", url = "https://ovextensionsprod.blob.core.windows.net/exts/kit/prod/107/shared" },
{ name = "kit/sdk", url = "https://ovextensionsprod.blob.core.windows.net/exts/kit/prod/sdk/${kit_version_short}/${kit_git_hash}" },
{ name = "kit/community", url = "https://dw290v42wisod.cloudfront.net/exts/kit/community" },
]
......@@ -156,6 +156,6 @@ folders = [
# set the S3 directory manually to the latest published S3
# note: this is done to ensure prior versions of Isaac Sim still use the latest assets
[settings]
persistent.isaac.asset_root.default = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0"
persistent.isaac.asset_root.cloud = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0"
persistent.isaac.asset_root.nvidia = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0"
persistent.isaac.asset_root.default = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1"
persistent.isaac.asset_root.cloud = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1"
persistent.isaac.asset_root.nvidia = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1"
apps/isaaclab.python.kit
View file @ 454c3d2f
......@@ -5,7 +5,7 @@
[package]
title = "Isaac Lab Python"
description = "An app for running Isaac Lab"
version = "2.2.1"
version = "2.3.0"
# That makes it browsable in UI with "experience" filter
keywords = ["experience", "app", "usd"]
......@@ -161,7 +161,7 @@ show_menu_titles = true
[settings.app]
name = "Isaac-Sim"
version = "5.0.0"
version = "5.1.0"
versionFile = "${exe-path}/VERSION"
content.emptyStageOnStart = true
fastShutdown = true
......@@ -255,7 +255,7 @@ outDirectory = "${data}"
###############################
[settings.exts."omni.kit.registry.nucleus"]
registries = [
{ name = "kit/default", url = "https://ovextensionsprod.blob.core.windows.net/exts/kit/prod/106/shared" },
{ name = "kit/default", url = "https://ovextensionsprod.blob.core.windows.net/exts/kit/prod/107/shared" },
{ name = "kit/sdk", url = "https://ovextensionsprod.blob.core.windows.net/exts/kit/prod/sdk/${kit_version_short}/${kit_git_hash}" },
{ name = "kit/community", url = "https://dw290v42wisod.cloudfront.net/exts/kit/community" },
]
......@@ -302,6 +302,6 @@ fabricUseGPUInterop = true
# set the S3 directory manually to the latest published S3
# note: this is done to ensure prior versions of Isaac Sim still use the latest assets
[settings]
persistent.isaac.asset_root.default = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0"
persistent.isaac.asset_root.cloud = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0"
persistent.isaac.asset_root.nvidia = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0"
persistent.isaac.asset_root.default = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1"
persistent.isaac.asset_root.cloud = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1"
persistent.isaac.asset_root.nvidia = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1"
apps/isaaclab.python.rendering.kit
View file @ 454c3d2f
......@@ -9,7 +9,7 @@
[package]
title = "Isaac Lab Python Camera"
description = "An app for running Isaac Lab with rendering enabled"
version = "2.2.1"
version = "2.3.0"
# That makes it browsable in UI with "experience" filter
keywords = ["experience", "app", "isaaclab", "python", "camera", "minimal"]
......@@ -33,7 +33,7 @@ cameras_enabled = true
app.versionFile = "${exe-path}/VERSION"
app.folder = "${exe-path}/"
app.name = "Isaac-Sim"
app.version = "5.0.0"
app.version = "5.1.0"
### FSD
app.useFabricSceneDelegate = true
......@@ -109,7 +109,7 @@ fabricUseGPUInterop = true
[settings.exts."omni.kit.registry.nucleus"]
registries = [
{ name = "kit/default", url = "https://ovextensionsprod.blob.core.windows.net/exts/kit/prod/106/shared" },
{ name = "kit/default", url = "https://ovextensionsprod.blob.core.windows.net/exts/kit/prod/107/shared" },
{ name = "kit/sdk", url = "https://ovextensionsprod.blob.core.windows.net/exts/kit/prod/sdk/${kit_version_short}/${kit_git_hash}" },
{ name = "kit/community", url = "https://dw290v42wisod.cloudfront.net/exts/kit/community" },
]
......@@ -145,6 +145,6 @@ folders = [
# set the S3 directory manually to the latest published S3
# note: this is done to ensure prior versions of Isaac Sim still use the latest assets
[settings]
persistent.isaac.asset_root.default = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0"
persistent.isaac.asset_root.cloud = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0"
persistent.isaac.asset_root.nvidia = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0"
persistent.isaac.asset_root.default = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1"
persistent.isaac.asset_root.cloud = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1"
persistent.isaac.asset_root.nvidia = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1"
apps/isaaclab.python.xr.openxr.headless.kit
View file @ 454c3d2f
......@@ -5,7 +5,7 @@
[package]
title = "Isaac Lab Python OpenXR Headless"
description = "An app for running Isaac Lab with OpenXR in headless mode"
version = "2.2.1"
version = "2.3.0"
# That makes it browsable in UI with "experience" filter
keywords = ["experience", "app", "usd", "headless"]
......@@ -15,16 +15,16 @@ keywords = ["experience", "app", "usd", "headless"]
app.versionFile = "${exe-path}/VERSION"
app.folder = "${exe-path}/"
app.name = "Isaac-Sim"
app.version = "5.0.0"
app.version = "5.1.0"
### FSD
app.useFabricSceneDelegate = true
# Temporary, should be enabled by default in Kit soon
rtx.hydra.readTransformsFromFabricInRenderDelegate = true
# work around for kitxr issue
app.hydra.renderSettings.useUsdAttributes = false
app.hydra.renderSettings.useFabricAttributes = false
# xr optimizations
xr.skipInputDeviceUSDWrites = true
'rtx-transient'.resourcemanager.enableTextureStreaming = false
[settings.isaaclab]
# This is used to check that this experience file is loaded when using cameras
......@@ -59,6 +59,6 @@ folders = [
]
[settings]
persistent.isaac.asset_root.default = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0"
persistent.isaac.asset_root.cloud = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0"
persistent.isaac.asset_root.nvidia = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0"
persistent.isaac.asset_root.default = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1"
persistent.isaac.asset_root.cloud = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1"
persistent.isaac.asset_root.nvidia = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1"
apps/isaaclab.python.xr.openxr.kit
View file @ 454c3d2f
......@@ -5,7 +5,7 @@
[package]
title = "Isaac Lab Python OpenXR"
description = "An app for running Isaac Lab with OpenXR"
version = "2.2.1"
version = "2.3.0"
# That makes it browsable in UI with "experience" filter
keywords = ["experience", "app", "usd"]
......@@ -15,7 +15,7 @@ keywords = ["experience", "app", "usd"]
app.versionFile = "${exe-path}/VERSION"
app.folder = "${exe-path}/"
app.name = "Isaac-Sim"
app.version = "5.0.0"
app.version = "5.1.0"
### async rendering settings
# omni.replicator.asyncRendering needs to be false for external camera rendering
......@@ -32,9 +32,9 @@ app.useFabricSceneDelegate = true
# Temporary, should be enabled by default in Kit soon
rtx.hydra.readTransformsFromFabricInRenderDelegate = true
# work around for kitxr issue
app.hydra.renderSettings.useUsdAttributes = false
app.hydra.renderSettings.useFabricAttributes = false
# xr optimizations
xr.skipInputDeviceUSDWrites = true
'rtx-transient'.resourcemanager.enableTextureStreaming = false
[dependencies]
"isaaclab.python" = {}
......@@ -88,6 +88,6 @@ folders = [
# set the S3 directory manually to the latest published S3
# note: this is done to ensure prior versions of Isaac Sim still use the latest assets
[settings]
persistent.isaac.asset_root.default = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0"
persistent.isaac.asset_root.cloud = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0"
persistent.isaac.asset_root.nvidia = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.0"
persistent.isaac.asset_root.default = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1"
persistent.isaac.asset_root.cloud = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1"
persistent.isaac.asset_root.nvidia = "https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1"
apps/isaacsim_4_5/isaaclab.python.headless.kit
View file @ 454c3d2f
......@@ -5,7 +5,7 @@
[package]
title = "Isaac Lab Python Headless"
description = "An app for running Isaac Lab headlessly"
version = "2.2.1"
version = "2.3.0"
# That makes it browsable in UI with "experience" filter
keywords = ["experience", "app", "isaaclab", "python", "headless"]
......
apps/isaacsim_4_5/isaaclab.python.headless.rendering.kit
View file @ 454c3d2f
......@@ -9,7 +9,7 @@
[package]
title = "Isaac Lab Python Headless Camera"
description = "An app for running Isaac Lab headlessly with rendering enabled"
version = "2.2.1"
version = "2.3.0"
# That makes it browsable in UI with "experience" filter
keywords = ["experience", "app", "isaaclab", "python", "camera", "minimal"]
......
apps/isaacsim_4_5/isaaclab.python.kit
View file @ 454c3d2f
......@@ -5,7 +5,7 @@
[package]
title = "Isaac Lab Python"
description = "An app for running Isaac Lab"
version = "2.2.1"
version = "2.3.0"
# That makes it browsable in UI with "experience" filter
keywords = ["experience", "app", "usd"]
......
apps/isaacsim_4_5/isaaclab.python.rendering.kit
View file @ 454c3d2f
......@@ -9,7 +9,7 @@
[package]
title = "Isaac Lab Python Camera"
description = "An app for running Isaac Lab with rendering enabled"
version = "2.2.1"
version = "2.3.0"
# That makes it browsable in UI with "experience" filter
keywords = ["experience", "app", "isaaclab", "python", "camera", "minimal"]
......
apps/isaacsim_4_5/isaaclab.python.xr.openxr.headless.kit
View file @ 454c3d2f
......@@ -5,7 +5,7 @@
[package]
title = "Isaac Lab Python OpenXR Headless"
description = "An app for running Isaac Lab with OpenXR in headless mode"
version = "2.2.1"
version = "2.3.0"
# That makes it browsable in UI with "experience" filter
keywords = ["experience", "app", "usd", "headless"]
......
apps/isaacsim_4_5/isaaclab.python.xr.openxr.kit
View file @ 454c3d2f
......@@ -5,7 +5,7 @@
[package]
title = "Isaac Lab Python OpenXR"
description = "An app for running Isaac Lab with OpenXR"
version = "2.2.1"
version = "2.3.0"
# That makes it browsable in UI with "experience" filter
keywords = ["experience", "app", "usd"]
......
docker/.env.base
View file @ 454c3d2f
......@@ -6,8 +6,8 @@
ACCEPT_EULA=Y
# NVIDIA Isaac Sim base image
ISAACSIM_BASE_IMAGE=nvcr.io/nvidia/isaac-sim
# NVIDIA Isaac Sim version to use (e.g. 5.0.0)
ISAACSIM_VERSION=5.0.0
# NVIDIA Isaac Sim version to use (e.g. 5.1.0)
ISAACSIM_VERSION=5.1.0
# Derived from the default path in the NVIDIA provided Isaac Sim container
DOCKER_ISAACSIM_ROOT_PATH=/isaac-sim
# The Isaac Lab path in the container
......
docker/.env.cloudxr-runtime
View file @ 454c3d2f
......@@ -5,4 +5,4 @@
# NVIDIA CloudXR Runtime base image
CLOUDXR_RUNTIME_BASE_IMAGE_ARG=nvcr.io/nvidia/cloudxr-runtime
# NVIDIA CloudXR Runtime version to use
CLOUDXR_RUNTIME_VERSION_ARG=5.0.0
CLOUDXR_RUNTIME_VERSION_ARG=5.0.1
docs/conf.py
View file @ 454c3d2f
......@@ -124,8 +124,8 @@ intersphinx_mapping = {
"python": ("https://docs.python.org/3", None),
"numpy": ("https://numpy.org/doc/stable/", None),
"trimesh": ("https://trimesh.org/", None),
"torch": ("https://docs.pytorch.org/docs/stable", None),
"isaacsim": ("https://docs.isaacsim.omniverse.nvidia.com/5.0.0/py/", None),
"torch": ("https://pytorch.org/docs/stable/", None),
"isaacsim": ("https://docs.isaacsim.omniverse.nvidia.com/5.1.0/py/", None),
"gymnasium": ("https://gymnasium.farama.org/", None),
"warp": ("https://nvidia.github.io/warp/", None),
"dev-guide": ("https://docs.omniverse.nvidia.com/dev-guide/latest", None),
......@@ -262,7 +262,7 @@ html_theme_options = {
{
"name": "Isaac Sim",
"url": "https://developer.nvidia.com/isaac-sim",
"icon": "https://img.shields.io/badge/IsaacSim-5.0.0-silver.svg",
"icon": "https://img.shields.io/badge/IsaacSim-5.1.0-silver.svg",
"type": "url",
},
{
......
docs/source/deployment/cloudxr_teleoperation_cluster.rst
View file @ 454c3d2f
......@@ -15,6 +15,9 @@ System Requirements
* **Minimum requirement**: Kubernetes cluster with a node that has at least 1 NVIDIA RTX PRO 6000 / L40 GPU or equivalent
* **Recommended requirement**: Kubernetes cluster with a node that has at least 2 RTX PRO 6000 / L40 GPUs or equivalent
.. note::
If you are using DGX Spark, check `DGX Spark Limitations <https://isaac-sim.github.io/IsaacLab/release/2.3.0/source/setup/installation/index.html#dgx-spark-details-and-limitations>`_ for compatibility.
Software Dependencies
---------------------
......@@ -76,7 +79,7 @@ Installation
.. code:: bash
helm fetch https://helm.ngc.nvidia.com/nvidia/charts/isaac-lab-teleop-2.2.0.tgz \
helm fetch https://helm.ngc.nvidia.com/nvidia/charts/isaac-lab-teleop-2.3.0.tgz \
--username='$oauthtoken' \
--password=<your-ngc-api-key>
......@@ -84,7 +87,7 @@ Installation
.. code:: bash
helm upgrade --install hello-isaac-teleop isaac-lab-teleop-2.2.0.tgz \
helm upgrade --install hello-isaac-teleop isaac-lab-teleop-2.3.0.tgz \
--set fullnameOverride=hello-isaac-teleop \
--set hostNetwork="true"
......@@ -107,7 +110,7 @@ Installation
# command
helm upgrade --install --values local_values.yml \
hello-isaac-teleop isaac-lab-teleop-2.2.0.tgz
hello-isaac-teleop isaac-lab-teleop-2.3.0.tgz
#. Verify the deployment is completed:
......
docs/source/deployment/docker.rst
View file @ 454c3d2f
......@@ -307,7 +307,7 @@ To pull the minimal Isaac Lab container, run:
.. code:: bash
docker pull nvcr.io/nvidia/isaac-lab:2.2.0
docker pull nvcr.io/nvidia/isaac-lab:2.3.0
To run the Isaac Lab container with an interactive bash session, run:
......@@ -323,7 +323,7 @@ To run the Isaac Lab container with an interactive bash session, run:
-v ~/docker/isaac-sim/logs:/root/.nvidia-omniverse/logs:rw \
-v ~/docker/isaac-sim/data:/root/.local/share/ov/data:rw \
-v ~/docker/isaac-sim/documents:/root/Documents:rw \
nvcr.io/nvidia/isaac-lab:2.2.0
nvcr.io/nvidia/isaac-lab:2.3.0
To enable rendering through X11 forwarding, run:
......@@ -342,7 +342,7 @@ To enable rendering through X11 forwarding, run:
-v ~/docker/isaac-sim/logs:/root/.nvidia-omniverse/logs:rw \
-v ~/docker/isaac-sim/data:/root/.local/share/ov/data:rw \
-v ~/docker/isaac-sim/documents:/root/Documents:rw \
nvcr.io/nvidia/isaac-lab:2.2.0
nvcr.io/nvidia/isaac-lab:2.3.0
To run an example within the container, run:
......
docs/source/experimental-features/newton-physics-integration/installation.rst
View file @ 454c3d2f
......@@ -44,7 +44,7 @@ Install the correct version of torch and torchvision:
.. code-block:: bash
pip install torch==2.7.0 torchvision==0.22.0 --index-url https://download.pytorch.org/whl/cu128
pip install -U torch==2.7.0 torchvision==0.22.0 --index-url https://download.pytorch.org/whl/cu128
Install Isaac Sim 5.0:
......
docs/source/features/ray.rst
View file @ 454c3d2f
......@@ -16,6 +16,13 @@ the general workflow is the same.
This functionality is experimental, and has been tested only on Linux.
.. warning::
**Security Notice**: Due to security risks associated with Ray,
this workflow is not intended for use outside of a strictly controlled
network environment. Ray clusters should only be deployed in trusted,
isolated networks with appropriate access controls and security measures in place.
Overview
......
docs/source/how-to/add_own_library.rst
View file @ 454c3d2f
......@@ -49,7 +49,7 @@ For instance, if you cloned the library to ``/home/user/git/rsl_rl``, the output
.. code-block:: bash
Name: rsl_rl
Version: 2.2.0
Version: 3.0.1
Summary: Fast and simple RL algorithms implemented in pytorch
Home-page: https://github.com/leggedrobotics/rsl_rl
Author: ETH Zurich, NVIDIA CORPORATION
......
docs/source/how-to/master_omniverse.rst
View file @ 454c3d2f
......@@ -68,7 +68,7 @@ Importing assets
- `Omniverse Create - Importing FBX Files \| NVIDIA Omniverse Tutorials <https://youtu.be/dQI0OpzfVHw>`__
- `Omniverse Asset Importer <https://docs.omniverse.nvidia.com/extensions/latest/ext_asset-importer.html>`__
- `Isaac Sim URDF impoter <https://docs.isaacsim.omniverse.nvidia.com/latest/robot_setup/ext_isaacsim_asset_importer_urdf.html>`__
- `Isaac Sim URDF impoter <https://docs.isaacsim.omniverse.nvidia.com/latest/importer_exporter/ext_isaacsim_asset_importer_urdf.html>`__
Part 2: Scripting in Omniverse
......
docs/source/how-to/simulation_performance.rst
View file @ 454c3d2f
Simulation Performance
=======================
Simulation Performance and Tuning
====================================
The performance of the simulation can be affected by various factors, including the number of objects in the scene,
the complexity of the physics simulation, and the hardware being used. Here are some tips to improve performance:
......@@ -61,5 +61,12 @@ CPU governors dictate the operating clock frequency range and scaling of the CPU
Additional Performance Guides
-----------------------------
There are many ways to "tune" the performance of the simulation, but the way you choose largely depends on what you are trying to simulate. In general, the first place
you will want to look for performance gains is with the `physics engine <https://docs.omniverse.nvidia.com/kit/docs/omni_physics/107.3/dev_guide/guides.html>`_. Next to rendering
and running deep learning models, the physics engine is the most computationally costly. Tuning the physics sim to limit the scope to only the task of interest is a great place to
start hunting for performance gains.
We have recently released a new `gripper tuning guide <https://docs.omniverse.nvidia.com/kit/docs/omni_physics/107.3/dev_guide/guides/gripper_tuning_example.html>`_ , specific to contact and grasp tuning. Please check it first if you intend to use robot grippers. For additional details, you should also checkout these guides!
* `Isaac Sim Performance Optimization Handbook <https://docs.isaacsim.omniverse.nvidia.com/latest/reference_material/sim_performance_optimization_handbook.html>`_
* `Omni Physics Simulation Performance Guide <https://docs.omniverse.nvidia.com/kit/docs/omni_physics/latest/dev_guide/guides/physics-performance.html>`_
docs/source/overview/environments.rst
View file @ 454c3d2f
......@@ -142,6 +142,16 @@ for the lift-cube environment:
| |gr1_pp_waist| | |gr1_pp_waist-link| | Pick up and place an object in a basket with a GR-1 humanoid robot |
| | | with waist degrees-of-freedom enables that provides a wider reach space. |
+-------------------------+------------------------------+-----------------------------------------------------------------------------+
| |g1_pick_place| | |g1_pick_place-link| | Pick up and place an object in a basket with a Unitree G1 humanoid robot |
+-------------------------+------------------------------+-----------------------------------------------------------------------------+
| |g1_pick_place_fixed| | |g1_pick_place_fixed-link| | Pick up and place an object in a basket with a Unitree G1 humanoid robot |
| | | with three-fingered hands. Robot is set up with the base fixed in place. |
+-------------------------+------------------------------+-----------------------------------------------------------------------------+
| |g1_pick_place_lm| | |g1_pick_place_lm-link| | Pick up and place an object in a basket with a Unitree G1 humanoid robot |
| | | with three-fingered hands and in-place locomanipulation capabilities |
| | | enabled (i.e. Robot lower body balances in-place while upper body is |
| | | controlled via Inverse Kinematics). |
+-------------------------+------------------------------+-----------------------------------------------------------------------------+
| |kuka-allegro-lift| | |kuka-allegro-lift-link| | Pick up a primitive shape on the table and lift it to target position |
+-------------------------+------------------------------+-----------------------------------------------------------------------------+
| |kuka-allegro-reorient| | |kuka-allegro-reorient-link| | Pick up a primitive shape on the table and orient it to target pose |
......@@ -163,6 +173,9 @@ for the lift-cube environment:
.. |cube-shadow| image:: ../_static/tasks/manipulation/shadow_cube.jpg
.. |stack-cube| image:: ../_static/tasks/manipulation/franka_stack.jpg
.. |gr1_pick_place| image:: ../_static/tasks/manipulation/gr-1_pick_place.jpg
.. |g1_pick_place| image:: ../_static/tasks/manipulation/g1_pick_place.jpg
.. |g1_pick_place_fixed| image:: ../_static/tasks/manipulation/g1_pick_place_fixed_base.jpg
.. |g1_pick_place_lm| image:: ../_static/tasks/manipulation/g1_pick_place_locomanipulation.jpg
.. |surface-gripper| image:: ../_static/tasks/manipulation/ur10_stack_surface_gripper.jpg
.. |gr1_pp_waist| image:: ../_static/tasks/manipulation/gr-1_pick_place_waist.jpg
.. |galbot_stack| image:: ../_static/tasks/manipulation/galbot_stack_cube.jpg
......@@ -184,6 +197,9 @@ for the lift-cube environment:
.. |stack-cube-link| replace:: `Isaac-Stack-Cube-Franka-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/stack/config/franka/stack_joint_pos_env_cfg.py>`__
.. |stack-cube-bp-link| replace:: `Isaac-Stack-Cube-Franka-IK-Rel-Blueprint-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/stack/config/franka/stack_ik_rel_blueprint_env_cfg.py>`__
.. |gr1_pick_place-link| replace:: `Isaac-PickPlace-GR1T2-Abs-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/pick_place/pickplace_gr1t2_env_cfg.py>`__
.. |g1_pick_place-link| replace:: `Isaac-PickPlace-G1-InspireFTP-Abs-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/pick_place/pickplace_unitree_g1_inspire_hand_env_cfg.py>`__
.. |g1_pick_place_fixed-link| replace:: `Isaac-PickPlace-FixedBaseUpperBodyIK-G1-Abs-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/isaaclab_tasks/isaaclab_tasks/manager_based/locomanipulation/pick_place/fixed_base_upper_body_ik_g1_env_cfg.py>`__
.. |g1_pick_place_lm-link| replace:: `Isaac-PickPlace-Locomanipulation-G1-Abs-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/isaaclab_tasks/isaaclab_tasks/manager_based/locomanipulation/pick_place/locomanipulation_g1_env_cfg.py>`__
.. |long-suction-link| replace:: `Isaac-Stack-Cube-UR10-Long-Suction-IK-Rel-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/stack/config/ur10_gripper/stack_ik_rel_env_cfg.py>`__
.. |short-suction-link| replace:: `Isaac-Stack-Cube-UR10-Short-Suction-IK-Rel-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/stack/config/ur10_gripper/stack_ik_rel_env_cfg.py>`__
.. |gr1_pp_waist-link| replace:: `Isaac-PickPlace-GR1T2-WaistEnabled-Abs-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/pick_place/pickplace_gr1t2_waist_enabled_env_cfg.py>`__
......@@ -197,6 +213,7 @@ for the lift-cube environment:
.. |agibot_place_mug-link| replace:: `Isaac-Place-Mug-Agibot-Left-Arm-RmpFlow-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/place/config/agibot/place_upright_mug_rmp_rel_env_cfg.py>`__
.. |agibot_place_toy-link| replace:: `Isaac-Place-Toy2Box-Agibot-Right-Arm-RmpFlow-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/place/config/agibot/place_toy2box_rmp_rel_env_cfg.py>`__
Contact-rich Manipulation
~~~~~~~~~~~~~~~~~~~~~~~~~
......@@ -973,6 +990,10 @@ inferencing, including reading from an already trained checkpoint and disabling
-
- Manager Based
-
* - Isaac-PickPlace-G1-InspireFTP-Abs-v0
-
- Manager Based
-
* - Isaac-Stack-Cube-UR10-Long-Suction-IK-Rel-v0
-
- Manager Based
......
docs/source/overview/imitation-learning/augmented_imitation.rst
View file @ 454c3d2f
......@@ -80,6 +80,8 @@ Example usage for the cube stacking task:
--input_file datasets/mimic_dataset_1k.hdf5 \
--output_dir datasets/mimic_dataset_1k_mp4
.. _running-cosmos:
Running Cosmos for Visual Augmentation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
......@@ -101,6 +103,9 @@ We provide an example augmentation output from `Cosmos Transfer1 <https://github
We recommend using the `Cosmos Transfer1 <https://github.com/nvidia-cosmos/cosmos-transfer1/tree/e4055e39ee9c53165e85275bdab84ed20909714a>`_ model for visual augmentation as we found it to produce the best results in the form of a highly diverse dataset with a wide range of visual variations. You can refer to the `installation instructions <https://github.com/nvidia-cosmos/cosmos-transfer1/blob/e4055e39ee9c53165e85275bdab84ed20909714a/INSTALL.md#environment-setup>`_, the `checkpoint download instructions <https://github.com/nvidia-cosmos/cosmos-transfer1/blob/e4055e39ee9c53165e85275bdab84ed20909714a/examples/inference_cosmos_transfer1_7b.md#download-checkpoints>`_ and `this example <https://github.com/nvidia-cosmos/cosmos-transfer1/blob/e4055e39ee9c53165e85275bdab84ed20909714a/examples/inference_cosmos_transfer1_7b.md#example-2-multimodal-control>`_ for reference on how to use Transfer1 for this usecase. We further recommend the following settings to be used with the Transfer1 model for this task:
.. note::
This workflow has been tested with commit ``e4055e39ee9c53165e85275bdab84ed20909714a`` of the Cosmos Transfer1 repository, and it is the recommended version to use. After cloning the Cosmos Transfer1 repository, checkout to this specific commit by running ``git checkout e4055e39ee9c53165e85275bdab84ed20909714a``.
.. rubric:: Hyperparameters
.. list-table::
......
docs/source/overview/imitation-learning/index.rst
View file @ 454c3d2f
......@@ -7,6 +7,6 @@ with Isaac Lab.
.. toctree::
:maxdepth: 1
augmented_imitation
teleop_imitation
augmented_imitation
skillgen
docs/source/overview/reinforcement-learning/rl_existing_scripts.rst
View file @ 454c3d2f
......@@ -8,6 +8,12 @@ from the environments into the respective libraries function argument and return
RL-Games
--------
.. attention::
When using RL-Games with the Ray workflow for distributed training or hyperparameter tuning,
please be aware that due to security risks associated with Ray, this workflow is not intended
for use outside of a strictly controlled network environment.
- Training an agent with
`RL-Games <https://github.com/Denys88/rl_games>`__ on ``Isaac-Ant-v0``:
......@@ -175,16 +181,16 @@ SKRL
Note that JAX GPU support is only available on Linux.
JAX 0.6.0 or higher (built on CuDNN v9.8) is incompatible with Isaac Lab's PyTorch 2.7 (built on CuDNN v9.7), and therefore not supported.
To install a compatible version of JAX for CUDA 12 use ``pip install "jax[cuda12]<0.6.0"``, for example.
To install a compatible version of JAX for CUDA 12 use ``pip install "jax[cuda12]<0.6.0" "flax<0.10.7"``, for example.
.. code:: bash
# install python module (for skrl)
./isaaclab.sh -i skrl
# install jax<0.6.0 for torch 2.7
./isaaclab.sh -p -m pip install "jax[cuda12]<0.6.0" "flax<0.10.7"
# install skrl dependencies for JAX
./isaaclab.sh -p -m pip install skrl["jax"]
# install jax<0.6.0 for torch 2.7
./isaaclab.sh -p -m pip install "jax[cuda12]<0.6.0"
# run script for training
./isaaclab.sh -p scripts/reinforcement_learning/skrl/train.py --task Isaac-Reach-Franka-v0 --headless --ml_framework jax
# run script for playing with 32 environments
......
docs/source/refs/issues.rst
View file @ 454c3d2f
......@@ -93,6 +93,17 @@ message and continue with terminating the process. On Windows systems, please us
``Ctrl+Break`` or ``Ctrl+fn+B`` to terminate the process.
URDF Importer: Unresolved references for fixed joints
-----------------------------------------------------
Starting with Isaac Sim 5.1, links connected through ``fixed_joint`` elements are no longer merged when
their URDF link entries specify mass and inertia even if ``merge-joint`` set to True.
This is expected behaviour—those links are treated as full bodies rather than zero-mass reference frames.
However, the USD importer currently raises ``ReportError`` warnings showing unresolved references for such links
when they lack visuals or colliders. This is a known bug in the importer; it creates references to visuals
that do not exist. The warnings can be safely ignored until the importer is updated.
GLIBCXX errors in Conda
-----------------------
......
docs/source/refs/reference_architecture/index.rst
View file @ 454c3d2f
.. _ref_arch:
Reference Architecture
======================
......
docs/source/refs/release_notes.rst
View file @ 454c3d2f
......@@ -1348,7 +1348,7 @@ Welcome to the first official release of Isaac Lab!
Building upon the foundation of the `Orbit <https://isaac-orbit.github.io/>`_ framework, we have integrated
the RL environment designing workflow from `OmniIsaacGymEnvs <https://github.com/NVIDIA-Omniverse/OmniIsaacGymEnvs>`_.
This allows users to choose a suitable `task-design approach <https://isaac-sim.github.io/IsaacLab/source/features/task_workflows.html>`_
This allows users to choose a suitable :ref:`task-design approach <ref_arch>`
for their applications.
While we maintain backward compatibility with Isaac Sim 2023.1.1, we highly recommend using Isaac Lab with
......
docs/source/setup/installation/cloud_installation.rst
View file @ 454c3d2f
......@@ -123,7 +123,7 @@ Next, run the deployment script for your preferred cloud:
.. note::
The ``--isaaclab`` flag is used to specify the version of Isaac Lab to deploy.
The ``v2.2.1`` tag is the latest release of Isaac Lab.
The ``v2.3.0`` tag is the latest release of Isaac Lab.
.. tab-set::
:sync-group: cloud
......@@ -133,28 +133,28 @@ Next, run the deployment script for your preferred cloud:
.. code-block:: bash
./deploy-aws --isaaclab v2.2.1
./deploy-aws --isaaclab v2.3.0
.. tab-item:: Azure
:sync: azure
.. code-block:: bash
./deploy-azure --isaaclab v2.2.1
./deploy-azure --isaaclab v2.3.0
.. tab-item:: GCP
:sync: gcp
.. code-block:: bash
./deploy-gcp --isaaclab v2.2.1
./deploy-gcp --isaaclab v2.3.0
.. tab-item:: Alibaba Cloud
:sync: alicloud
.. code-block:: bash
./deploy-alicloud --isaaclab v2.2.1
./deploy-alicloud --isaaclab v2.3.0
Follow the prompts for entering information regarding the environment setup and credentials.
Once successful, instructions for connecting to the cloud instance will be available
......
docs/source/setup/installation/include/bin_verify_isaacsim.rst
View file @ 454c3d2f
......@@ -71,4 +71,4 @@ instructions, it means that something is incorrectly configured. To
debug and troubleshoot, please check Isaac Sim
`documentation <https://docs.omniverse.nvidia.com/dev-guide/latest/linux-troubleshooting.html>`__
and the
`forums <https://docs.isaacsim.omniverse.nvidia.com/latest/isaac_sim_forums.html>`__.
`Isaac Sim Forums <https://docs.isaacsim.omniverse.nvidia.com/latest/common/feedback.html>`_.
docs/source/setup/installation/include/pip_verify_isaacsim.rst
View file @ 454c3d2f
......@@ -43,4 +43,4 @@ instructions, it means that something is incorrectly configured. To
debug and troubleshoot, please check Isaac Sim
`documentation <https://docs.omniverse.nvidia.com/dev-guide/latest/linux-troubleshooting.html>`__
and the
`forums <https://docs.isaacsim.omniverse.nvidia.com//latest/isaac_sim_forums.html>`__.
`Isaac Sim Forums <https://docs.isaacsim.omniverse.nvidia.com/latest/common/feedback.html>`_.
docs/source/setup/installation/include/src_python_virtual_env.rst
View file @ 454c3d2f
......@@ -67,7 +67,7 @@ instead of *./isaaclab.sh -p* or *isaaclab.bat -p*.
.. tab-item:: Conda Environment
To install conda, please follow the instructions `here <https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html>__`.
To install conda, please follow the instructions `here <https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html>`__.
You can create the Isaac Lab environment using the following commands.
We recommend using `Miniconda <https://www.anaconda.com/docs/getting-started/miniconda/main/>`_,
......
docs/source/setup/installation/include/src_verify_isaaclab.rst
View file @ 454c3d2f
......@@ -93,7 +93,8 @@ We recommend adding ``--headless`` for faster training.
isaaclab.bat -p scripts/reinforcement_learning/rsl_rl/train.py --task=Isaac-Velocity-Rough-Anymal-C-v0 --headless
Isaac Lab provides the tools you'll need to create your own **Tasks** and **Workflows** for whatever your project needs may be. Take a look at our :ref:`how-to` guides like `Adding your own learning Library <source/how-to/add_own_library>`_ or `Wrapping Environments <source/how-to/wrap_rl_env>`_ for details.
Isaac Lab provides the tools you'll need to create your own **Tasks** and **Workflows** for whatever your project needs may be.
Take a look at our :ref:`how-to` guides like :ref:`Adding your own learning Library <how-to-add-library>` or :ref:`Wrapping Environments <how-to-env-wrappers>` for details.
.. figure:: /source/_static/setup/isaac_ants_example.jpg
:align: center
......
docs/source/setup/installation/index.rst
View file @ 454c3d2f
......@@ -3,9 +3,9 @@
Local Installation
==================
.. image:: https://img.shields.io/badge/IsaacSim-5.0.0-silver.svg
.. image:: https://img.shields.io/badge/IsaacSim-5.1.0-silver.svg
:target: https://developer.nvidia.com/isaac-sim
:alt: IsaacSim 5.0.0
:alt: IsaacSim 5.1.0
.. image:: https://img.shields.io/badge/python-3.11-blue.svg
:target: https://www.python.org/downloads/release/python-31013/
......@@ -27,7 +27,7 @@ recommended installation methods for both Isaac Sim and Isaac Lab.
.. caution::
We have dropped support for Isaac Sim versions 4.2.0 and below. We recommend using the latest
Isaac Sim 5.0.0 release to benefit from the latest features and improvements.
Isaac Sim 5.1.0 release to benefit from the latest features and improvements.
For more information, please refer to the
`Isaac Sim release notes <https://docs.isaacsim.omniverse.nvidia.com/latest/overview/release_notes.html#>`__.
......@@ -68,6 +68,31 @@ may work but have not been validated against all Omniverse tests.
driver from the `Unix Driver Archive <https://www.nvidia.com/en-us/drivers/unix/>`_
using the ``.run`` installer.
DGX Spark: details and limitations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The DGX spark is a standalone machine learning device with aarch64 architecture. As a consequence, some
features of Isaac Lab are not currently supported on the DGX spark. The most noteworthy is that the architecture *requires* CUDA ≥ 13, and thus the cu13 build of PyTorch or newer.
Other notable limitations with respect to Isaac Lab include...
#. `SkillGen <https://isaac-sim.github.io/IsaacLab/main/source/overview/imitation-learning/skillgen.html>`_ is not supported out of the box. This
is because cuRobo builds native CUDA/C++ extensions that requires specific tooling and library versions which are not validated for use with DGX spark.
#. Extended reality teleoperation tools such as `OpenXR <https://isaac-sim.github.io/IsaacLab/release/2.3.0/source/api/lab/isaaclab.devices.html#openxr>`_ is not supported. This is due
to encoding performance limitations that have not yet been fully investigated.
#. SKRL training with JAX <https://docs.jax.dev/en/latest/notebooks/thinking_in_jax.html>_ has not been explicitly validated or tested in Isaac Lab on the DGX Spark.
JAX provides pre-built CUDA wheels only for Linux on x86_64, so on aarch64 systems (e.g., DGX Spark) it runs on CPU only by default.
GPU support requires building JAX from source, which has not been validated in Isaac Lab.
#. Livestream and Hub Workstation Cache are not supported on the DGX spark.
#. Multi-node training is not currently supported.
#. :ref:`Isaac Lab Mimic <generating-additional-demonstrations>` data generation and policy inference for visuomotor envrionements are not supported on DGX Spark due to a lack of non-DLSS image denoiser on aarch64.
#. :ref:`Running Cosmos Transfer1 <running-cosmos>` is not currently supported on the DGX Spark.
Troubleshooting
~~~~~~~~~~~~~~~
......
docs/source/setup/installation/isaaclab_pip_installation.rst
View file @ 454c3d2f
......@@ -44,6 +44,68 @@ Installing dependencies
pip install isaaclab[isaacsim,all]==2.2.0 --extra-index-url https://pypi.nvidia.com
In case you used UV to create your virtual environment, please replace ``pip`` with ``uv pip``
in the following commands.
- Install the Isaac Lab packages along with Isaac Sim:
.. code-block:: none
pip install isaaclab[isaacsim,all]==2.3.0 --extra-index-url https://pypi.nvidia.com
- Install a CUDA-enabled PyTorch build that matches your system architecture:
.. tab-set::
:sync-group: pip-platform
.. tab-item:: :icon:`fa-brands fa-linux` Linux (x86_64)
:sync: linux-x86_64
.. code-block:: bash
pip install -U torch==2.7.0 torchvision==0.22.0 --index-url https://download.pytorch.org/whl/cu128
.. tab-item:: :icon:`fa-brands fa-windows` Windows (x86_64)
:sync: windows-x86_64
.. code-block:: bash
pip install -U torch==2.7.0 torchvision==0.22.0 --index-url https://download.pytorch.org/whl/cu128
.. tab-item:: :icon:`fa-brands fa-linux` Linux (aarch64)
:sync: linux-aarch64
.. code-block:: bash
pip install -U torch==2.9.0 torchvision==0.24.0 --index-url https://download.pytorch.org/whl/cu130
.. note::
After installing Isaac Lab on aarch64, you may encounter warnings such as:
.. code-block:: none
ERROR: ld.so: object '...torch.libs/libgomp-XXXX.so.1.0.0' cannot be preloaded: ignored.
This occurs when both the system and PyTorch ``libgomp`` (GNU OpenMP) libraries are preloaded.
Isaac Sim expects the **system** OpenMP runtime, while PyTorch sometimes bundles its own.
To fix this, unset any existing ``LD_PRELOAD`` and set it to use the system library only:
.. code-block:: bash
unset LD_PRELOAD
export LD_PRELOAD="$LD_PRELOAD:/lib/aarch64-linux-gnu/libgomp.so.1"
This ensures the correct ``libgomp`` library is preloaded for both Isaac Sim and Isaac Lab,
removing the preload warnings during runtime.
- If you want to use ``rl_games`` for training and inferencing, install
its Python 3.11 enabled fork:
.. code-block:: none
pip install git+https://github.com/isaac-sim/rl_games.git@python3.11
.. include:: include/pip_verify_isaacsim.rst
......
docs/source/setup/installation/pip_installation.rst
View file @ 454c3d2f
......@@ -42,17 +42,58 @@ Installing dependencies
In case you used UV to create your virtual environment, please replace ``pip`` with ``uv pip``
in the following commands.
- Install a CUDA-enabled PyTorch 2.7.0 build for CUDA 12.8:
- Install Isaac Sim pip packages:
.. code-block:: bash
.. code-block:: none
pip install torch==2.7.0 torchvision==0.22.0 --index-url https://download.pytorch.org/whl/cu128
pip install "isaacsim[all,extscache]==5.1.0" --extra-index-url https://pypi.nvidia.com
- Install Isaac Sim pip packages:
- Install a CUDA-enabled PyTorch build that matches your system architecture:
.. code-block:: none
.. tab-set::
:sync-group: pip-platform
.. tab-item:: :icon:`fa-brands fa-linux` Linux (x86_64)
:sync: linux-x86_64
.. code-block:: bash
pip install -U torch==2.7.0 torchvision==0.22.0 --index-url https://download.pytorch.org/whl/cu128
.. tab-item:: :icon:`fa-brands fa-windows` Windows (x86_64)
:sync: windows-x86_64
.. code-block:: bash
pip install -U torch==2.7.0 torchvision==0.22.0 --index-url https://download.pytorch.org/whl/cu128
.. tab-item:: :icon:`fa-brands fa-linux` Linux (aarch64)
:sync: linux-aarch64
.. code-block:: bash
pip install -U torch==2.9.0 torchvision==0.24.0 --index-url https://download.pytorch.org/whl/cu130
.. note::
After installing Isaac Lab on aarch64, you may encounter warnings such as:
.. code-block:: none
ERROR: ld.so: object '...torch.libs/libgomp-XXXX.so.1.0.0' cannot be preloaded: ignored.
This occurs when both the system and PyTorch ``libgomp`` (GNU OpenMP) libraries are preloaded.
Isaac Sim expects the **system** OpenMP runtime, while PyTorch sometimes bundles its own.
To fix this, unset any existing ``LD_PRELOAD`` and set it to use the system library only:
.. code-block:: bash
unset LD_PRELOAD
export LD_PRELOAD="$LD_PRELOAD:/lib/aarch64-linux-gnu/libgomp.so.1"
pip install "isaacsim[all,extscache]==5.0.0" --extra-index-url https://pypi.nvidia.com
This ensures the correct ``libgomp`` library is preloaded for both Isaac Sim and Isaac Lab,
removing the preload warnings during runtime.
.. include:: include/pip_verify_isaacsim.rst
......
docs/source/setup/quickstart.rst
View file @ 454c3d2f
......@@ -70,7 +70,7 @@ Next, install a CUDA-enabled PyTorch 2.7.0 build.
.. code-block:: bash
pip install torch==2.7.0 torchvision==0.22.0 --index-url https://download.pytorch.org/whl/cu128
pip install -U torch==2.7.0 torchvision==0.22.0 --index-url https://download.pytorch.org/whl/cu128
Before we can install Isaac Sim, we need to make sure pip is updated. To update pip, run
......@@ -96,7 +96,7 @@ and now we can install the Isaac Sim packages.
.. code-block:: none
pip install "isaacsim[all,extscache]==5.0.0" --extra-index-url https://pypi.nvidia.com
pip install "isaacsim[all,extscache]==5.1.0" --extra-index-url https://pypi.nvidia.com
Finally, we can install Isaac Lab. To start, clone the repository using the following
......
isaaclab.bat
View file @ 454c3d2f
......@@ -511,14 +511,8 @@ if "%arg%"=="-i" (
call :extract_python_exe
echo [INFO] Using python from: !python_exe!
REM Loop through all arguments - mimic shift
set "allArgs="
for %%a in (%*) do (
REM Append each argument to the variable, skip the first one
if defined skip (
set "allArgs=!allArgs! %%a"
) else (
set "skip=1"
)
for /f "tokens=1,* delims= " %%a in ("%*") do (
set "allArgs=%%b"
)
call !python_exe! !allArgs!
goto :end
......@@ -527,14 +521,8 @@ if "%arg%"=="-i" (
call :extract_python_exe
echo [INFO] Using python from: !python_exe!
REM Loop through all arguments - mimic shift
set "allArgs="
for %%a in (%*) do (
REM Append each argument to the variable, skip the first one
if defined skip (
set "allArgs=!allArgs! %%a"
) else (
set "skip=1"
)
for /f "tokens=1,* delims= " %%a in ("%*") do (
set "allArgs=%%b"
)
call !python_exe! !allArgs!
goto :end
......
isaaclab.sh
View file @ 454c3d2f
......@@ -96,29 +96,47 @@ is_docker() {
[[ "$(hostname)" == *"."* ]]
}
# check if running on ARM architecture
is_arm() {
[[ "$(uname -m)" == "aarch64" ]] || [[ "$(uname -m)" == "arm64" ]]
}
ensure_cuda_torch() {
local pip_command=$(extract_pip_command)
local pip_uninstall_command=$(extract_pip_uninstall_command)
local -r TORCH_VER="2.7.0"
local -r TV_VER="0.22.0"
local -r CUDA_TAG="cu128"
local -r PYTORCH_INDEX="https://download.pytorch.org/whl/${CUDA_TAG}"
local torch_ver
if "$pip_command" show torch >/dev/null 2>&1; then
torch_ver="$("$pip_command" show torch 2>/dev/null | awk -F': ' '/^Version/{print $2}')"
echo "[INFO] Found PyTorch version ${torch_ver}."
if [[ "$torch_ver" != "${TORCH_VER}+${CUDA_TAG}" ]]; then
echo "[INFO] Replacing PyTorch ${torch_ver} → ${TORCH_VER}+${CUDA_TAG}..."
"$pip_uninstall_command" torch torchvision torchaudio >/dev/null 2>&1 || true
"$pip_command" "torch==${TORCH_VER}" "torchvision==${TV_VER}" --index-url "${PYTORCH_INDEX}"
local py="$1"
# base base index for torch
local base_index="https://download.pytorch.org/whl"
# choose pins per arch
local torch_ver tv_ver cuda_ver
if is_arm; then
torch_ver="2.9.0"
tv_ver="0.24.0"
cuda_ver="130"
else
echo "[INFO] PyTorch ${TORCH_VER}+${CUDA_TAG} already installed."
torch_ver="2.7.0"
tv_ver="0.22.0"
cuda_ver="128"
fi
else
echo "[INFO] Installing PyTorch ${TORCH_VER}+${CUDA_TAG}..."
${pip_command} "torch==${TORCH_VER}" "torchvision==${TV_VER}" --index-url "${PYTORCH_INDEX}"
fi
local index="${base_index}/cu${cuda_ver}"
local want_torch="${torch_ver}+cu${cuda_ver}"
# check current torch version (may be empty)
local cur=""
if "$py" -m pip show torch >/dev/null 2>&1; then
cur="$("$py" -m pip show torch 2>/dev/null | awk -F': ' '/^Version/{print $2}')"
fi
# skip install if version is already satisfied
if [[ "$cur" == "$want_torch" ]]; then
return 0
fi
# clean install torch
echo "[INFO] Installing torch==${torch_ver} and torchvision==${tv_ver} (cu${cuda_ver}) from ${index}..."
"$py" -m pip uninstall -y torch torchvision torchaudio >/dev/null 2>&1 || true
"$py" -m pip install -U --index-url "${index}" "torch==${torch_ver}" "torchvision==${tv_ver}"
}
# extract isaac sim path
......@@ -286,6 +304,26 @@ fi
EOS
}
# Temporarily unset LD_PRELOAD (ARM only) for a block of commands
begin_arm_install_sandbox() {
if is_arm && [[ -n "${LD_PRELOAD:-}" ]]; then
export _IL_SAVED_LD_PRELOAD="$LD_PRELOAD"
unset LD_PRELOAD
echo "[INFO] ARM install sandbox: temporarily unsetting LD_PRELOAD for installation."
fi
# ensure we restore even if a command fails (set -e)
trap 'end_arm_install_sandbox' EXIT
}
end_arm_install_sandbox() {
if [[ -n "${_IL_SAVED_LD_PRELOAD:-}" ]]; then
export LD_PRELOAD="$_IL_SAVED_LD_PRELOAD"
unset _IL_SAVED_LD_PRELOAD
fi
# remove trap so later exits don’t re-run restore
trap - EXIT
}
# setup anaconda environment for Isaac Lab
setup_conda_env() {
# get environment name from input
......@@ -317,7 +355,7 @@ setup_conda_env() {
echo "[INFO] Detected Isaac Sim 4.5 → forcing python=3.10"
sed -i 's/^ - python=3\.11/ - python=3.10/' "${ISAACLAB_PATH}/environment.yml"
else
echo "[INFO] Isaac Sim 5.0, installing python=3.11"
echo "[INFO] Isaac Sim >= 5.0 detected, installing python=3.11"
fi
conda env create -y --file ${ISAACLAB_PATH}/environment.yml -n ${env_name}
......@@ -525,9 +563,12 @@ while [[ $# -gt 0 ]]; do
pip_command=$(extract_pip_command)
pip_uninstall_command=$(extract_pip_uninstall_command)
# check if pytorch is installed and its version
# install pytorch with cuda 12.8 for blackwell support
ensure_cuda_torch
# if on ARM arch, temporarily clear LD_PRELOAD
# LD_PRELOAD is restored below, after installation
begin_arm_install_sandbox
# install pytorch (version based on arch)
ensure_cuda_torch ${python_exe}
# recursively look into directories and install them
# this does not check dependencies between extensions
export -f extract_python_exe
......@@ -557,7 +598,11 @@ while [[ $# -gt 0 ]]; do
# in some rare cases, torch might not be installed properly by setup.py, add one more check here
# can prevent that from happening
ensure_cuda_torch
ensure_cuda_torch ${python_exe}
# restore LD_PRELOAD if we cleared it
end_arm_install_sandbox
# check if we are inside a docker container or are building a docker image
# in that case don't setup VSCode since it asks for EULA agreement which triggers user interaction
if is_docker; then
......@@ -628,11 +673,16 @@ while [[ $# -gt 0 ]]; do
if [ -n "${CONDA_DEFAULT_ENV}" ] || [ -n "${VIRTUAL_ENV}" ]; then
export PYTHONPATH=${cache_pythonpath}
fi
shift # past argument
# exit neatly
break
;;
-p|--python)
# ensures Kit loads Isaac Sim’s icon instead of a generic icon on aarch64
if is_arm; then
export RESOURCE_NAME="${RESOURCE_NAME:-IsaacSim}"
fi
# run the python provided by isaacsim
python_exe=$(extract_python_exe)
echo "[INFO] Using python from: ${python_exe}"
......
scripts/demos/sensors/raycaster_sensor.py
View file @ 454c3d2f
......@@ -4,7 +4,6 @@
# SPDX-License-Identifier: BSD-3-Clause
import argparse
import numpy as np
from isaaclab.app import AppLauncher
......@@ -22,6 +21,7 @@ simulation_app = app_launcher.app
"""Rest everything follows."""
import numpy as np
import torch
import isaaclab.sim as sim_utils
......
scripts/environments/teleoperation/teleop_se3_agent.py
View file @ 454c3d2f
......@@ -19,7 +19,11 @@ parser.add_argument(
"--teleop_device",
type=str,
default="keyboard",
help="Device for interacting with environment. Examples: keyboard, spacemouse, gamepad, handtracking, manusvive",
help=(
"Teleop device. Set here (legacy) or via the environment config. If using the environment config, pass the"
" device key/name defined under 'teleop_devices' (it can be a custom name, not necessarily 'handtracking')."
" Built-ins: keyboard, spacemouse, gamepad. Not all tasks support all built-ins."
),
)
parser.add_argument("--task", type=str, default=None, help="Name of the task.")
parser.add_argument("--sensitivity", type=float, default=1.0, help="Sensitivity factor.")
......@@ -66,6 +70,7 @@ from isaaclab_tasks.manager_based.manipulation.lift import mdp
from isaaclab_tasks.utils import parse_env_cfg
if args_cli.enable_pinocchio:
import isaaclab_tasks.manager_based.locomanipulation.pick_place # noqa: F401
import isaaclab_tasks.manager_based.manipulation.pick_place # noqa: F401
......
scripts/imitation_learning/locomanipulation_sdg/plot_navigation_trajectory.py 0 → 100644
View file @ 454c3d2f
# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
"""Script to visualize navigation datasets.
Loads a navigation dataset and generates plots showing paths, poses and obstacles.
Args:
dataset: Path to the HDF5 dataset file containing recorded demonstrations.
output_dir: Directory path where visualization plots will be saved.
figure_size: Size of the generated figures (width, height).
demo_filter: If provided, only visualize specific demo(s). Can be a single demo name or comma-separated list.
"""
import argparse
import h5py
import matplotlib.pyplot as plt
import os
def main():
"""Main function to process dataset and generate visualizations."""
# add argparse arguments
parser = argparse.ArgumentParser(
description="Visualize navigation dataset from locomanipulation sdg demonstrations."
)
parser.add_argument(
"--input_file", type=str, help="Path to the HDF5 dataset file containing recorded demonstrations."
)
parser.add_argument("--output_dir", type=str, help="Directory path where visualization plots will be saved.")
parser.add_argument(
"--figure_size",
type=int,
nargs=2,
default=[20, 20],
help="Size of the generated figures (width, height). Default: [20, 20]",
)
parser.add_argument(
"--demo_filter",
type=str,
default=None,
help="If provided, only visualize specific demo(s). Can be a single demo name or comma-separated list.",
)
# parse the arguments
args = parser.parse_args()
# Validate inputs
if not os.path.exists(args.input_file):
raise FileNotFoundError(f"Dataset file not found: {args.input_file}")
# Create output directory if it doesn't exist
os.makedirs(args.output_dir, exist_ok=True)
# Load dataset
dataset = h5py.File(args.input_file, "r")
demos = list(dataset["data"].keys())
# Filter demos if specified
if args.demo_filter:
filter_demos = [d.strip() for d in args.demo_filter.split(",")]
demos = [d for d in demos if d in filter_demos]
if not demos:
print(f"Warning: No demos found matching filter '{args.demo_filter}'")
return
print(f"Visualizing {len(demos)} demonstrations...")
for i, demo in enumerate(demos):
print(f"Processing demo {i + 1}/{len(demos)}: {demo}")
replay_data = dataset["data"][demo]["locomanipulation_sdg_output_data"]
path = replay_data["base_path"]
base_pose = replay_data["base_pose"]
object_pose = replay_data["object_pose"]
start_pose = replay_data["start_fixture_pose"]
end_pose = replay_data["end_fixture_pose"]
obstacle_poses = replay_data["obstacle_fixture_poses"]
plt.figure(figsize=args.figure_size)
plt.plot(path[0, :, 0], path[0, :, 1], "r-", label="Target Path", linewidth=2)
plt.plot(base_pose[:, 0], base_pose[:, 1], "g--", label="Base Pose", linewidth=2)
plt.plot(object_pose[:, 0], object_pose[:, 1], "b--", label="Object Pose", linewidth=2)
plt.plot(obstacle_poses[0, :, 0], obstacle_poses[0, :, 1], "ro", label="Obstacles", markersize=8)
# Add start and end markers
plt.plot(start_pose[0, 0], start_pose[0, 1], "gs", label="Start", markersize=12)
plt.plot(end_pose[0, 0], end_pose[0, 1], "rs", label="End", markersize=12)
plt.legend(loc="upper right", ncol=1, fontsize=12)
plt.axis("equal")
plt.grid(True, alpha=0.3)
plt.title(f"Navigation Visualization - {demo}", fontsize=16)
plt.xlabel("X Position (m)", fontsize=14)
plt.ylabel("Y Position (m)", fontsize=14)
output_path = os.path.join(args.output_dir, f"{demo}.png")
plt.savefig(output_path, dpi=150, bbox_inches="tight")
plt.close() # Close the figure to free memory
dataset.close()
print(f"Visualization complete! Plots saved to: {args.output_dir}")
if __name__ == "__main__":
main()
scripts/imitation_learning/robomimic/play.py
View file @ 454c3d2f
......@@ -70,6 +70,7 @@ import robomimic.utils.torch_utils as TorchUtils
if args_cli.enable_pinocchio:
import isaaclab_tasks.manager_based.manipulation.pick_place # noqa: F401
import isaaclab_tasks.manager_based.locomanipulation.pick_place # noqa: F401
from isaaclab_tasks.utils import parse_env_cfg
......
scripts/imitation_learning/robomimic/train.py
View file @ 454c3d2f
......@@ -84,6 +84,7 @@ from robomimic.utils.log_utils import DataLogger, PrintLogger
# Isaac Lab imports (needed so that environment is registered)
import isaaclab_tasks # noqa: F401
import isaaclab_tasks.manager_based.locomanipulation.pick_place # noqa: F401
import isaaclab_tasks.manager_based.manipulation.pick_place # noqa: F401
......
scripts/reinforcement_learning/rl_games/train.py
View file @ 454c3d2f
......@@ -174,7 +174,7 @@ def main(env_cfg: ManagerBasedRLEnvCfg | DirectRLEnvCfg | DirectMARLEnvCfg, agen
)
# set the log directory for the environment (works for all environment types)
env_cfg.log_dir = log_dir
env_cfg.log_dir = os.path.join(log_root_path, log_dir)
# create isaac environment
env = gym.make(args_cli.task, cfg=env_cfg, render_mode="rgb_array" if args_cli.video else None)
......
scripts/tools/record_demos.py
View file @ 454c3d2f
......@@ -33,7 +33,16 @@ from isaaclab.app import AppLauncher
# add argparse arguments
parser = argparse.ArgumentParser(description="Record demonstrations for Isaac Lab environments.")
parser.add_argument("--task", type=str, required=True, help="Name of the task.")
parser.add_argument("--teleop_device", type=str, default="keyboard", help="Device for interacting with environment.")
parser.add_argument(
"--teleop_device",
type=str,
default="keyboard",
help=(
"Teleop device. Set here (legacy) or via the environment config. If using the environment config, pass the"
" device key/name defined under 'teleop_devices' (it can be a custom name, not necessarily 'handtracking')."
" Built-ins: keyboard, spacemouse, gamepad. Not all tasks support all built-ins."
),
)
parser.add_argument(
"--dataset_file", type=str, default="./datasets/dataset.hdf5", help="File path to export recorded demos."
)
......@@ -98,6 +107,7 @@ from isaaclab_mimic.ui.instruction_display import InstructionDisplay, show_subta
if args_cli.enable_pinocchio:
import isaaclab_tasks.manager_based.manipulation.pick_place # noqa: F401
import isaaclab_tasks.manager_based.locomanipulation.pick_place # noqa: F401
from collections.abc import Callable
......@@ -304,7 +314,7 @@ def setup_ui(label_text: str, env: gym.Env) -> InstructionDisplay:
Returns:
InstructionDisplay: The configured instruction display object
"""
instruction_display = InstructionDisplay(args_cli.teleop_device)
instruction_display = InstructionDisplay(args_cli.xr)
if not args_cli.xr:
window = EmptyWindow(env, "Instruction")
with window.ui_window_elements["main_vstack"]:
......
scripts/tools/replay_demos.py
View file @ 454c3d2f
......@@ -66,6 +66,7 @@ from isaaclab.utils.datasets import EpisodeData, HDF5DatasetFileHandler
if args_cli.enable_pinocchio:
import isaaclab_tasks.manager_based.manipulation.pick_place # noqa: F401
import isaaclab_tasks.manager_based.locomanipulation.pick_place # noqa: F401
import isaaclab_tasks # noqa: F401
from isaaclab_tasks.utils.parse_cfg import parse_env_cfg
......
source/isaaclab/config/extension.toml
View file @ 454c3d2f
[package]
# Note: Semantic Versioning is used: https://semver.org/
version = "0.47.2"
version = "0.47.3"
# Description
title = "Isaac Lab framework for Robot Learning"
......
source/isaaclab/docs/CHANGELOG.rst
View file @ 454c3d2f
Changelog
---------
0.47.2 (2025-10-22)
0.47.3 (2025-10-22)
~~~~~~~~~~~~~~~~~~~
Changed
......@@ -11,7 +11,7 @@ Changed
support the correct data type when converting from numpy arrays to warp arrays on the CPU.
0.47.1 (2025-10-17)
0.47.2 (2025-10-17)
~~~~~~~~~~~~~~~~~~~
Added
......@@ -21,6 +21,17 @@ Added
* Added :meth:`~isaaclab.sim.utils.resolve_prim_scale` to resolve the scale of a prim in the world frame.
0.47.1 (2025-10-17)
~~~~~~~~~~~~~~~~~~~
Fixed
^^^^^
* Suppressed yourdfpy warnings when trying to load meshes from hand urdfs in dex_retargeting. These mesh files are not
used by dex_retargeting, but the parser is incorrectly configured by dex_retargeting to load them anyway which results
in warning spam.
0.47.0 (2025-10-14)
~~~~~~~~~~~~~~~~~~~
......@@ -31,6 +42,71 @@ Changed
Configurations can continue to be saved and loaded through yaml.
0.46.11 (2025-10-15)
~~~~~~~~~~~~~~~~~~~~
Added
^^^^^
* Added support for modifying the :attr:`/rtx/domeLight/upperLowerStrategy` Sim rendering setting.
0.46.10 (2025-10-13)
~~~~~~~~~~~~~~~~~~~~
Added
^^^^^
* Added ARM64 architecture for pink ik and dex-retargetting setup installations.
0.46.9 (2025-10-09)
~~~~~~~~~~~~~~~~~~~
Fixed
^^^^^
* Fixed :meth:`~isaaclab.devices.keyboard.se3_keyboard.Se3Keyboard.__del__` to use the correct method name
for unsubscribing from keyboard events "unsubscribe_to_keyboard_events" instead of "unsubscribe_from_keyboard_events".
0.46.8 (2025-10-02)
~~~~~~~~~~~~~~~~~~~
Fixed
^^^^^
* Fixed scaling factor for retargeting of GR1T2 hand.
0.46.7 (2025-09-30)
~~~~~~~~~~~~~~~~~~~
Fixed
^^^^^
* Fixed finger joint indices with manus extension.
0.46.6 (2025-09-30)
~~~~~~~~~~~~~~~~~~~
Added
^^^^^
* Added argument :attr:`traverse_instance_prims` to :meth:`~isaaclab.sim.utils.get_all_matching_child_prims` and
:meth:`~isaaclab.sim.utils.get_first_matching_child_prim` to control whether to traverse instance prims
during the traversal. Earlier, instanced prims were skipped since :meth:`Usd.Prim.GetChildren` did not return
instanced prims, which is now fixed.
Changed
^^^^^^^
* Made parsing of instanced prims in :meth:`~isaaclab.sim.utils.get_all_matching_child_prims` and
:meth:`~isaaclab.sim.utils.get_first_matching_child_prim` as the default behavior.
* Added parsing of instanced prims in :meth:`~isaaclab.sim.utils.make_uninstanceable` to make all prims uninstanceable.
0.46.5 (2025-10-14)
~~~~~~~~~~~~~~~~~~~
......@@ -100,6 +176,16 @@ Changed
* Added parsing of instanced prims in :meth:`~isaaclab.sim.utils.make_uninstanceable` to make all prims uninstanceable.
0.45.16 (2025-09-06)
~~~~~~~~~~~~~~~~~~~~
Added
^^^^^
* Added teleoperation environments for Unitree G1. This includes an environment with lower body fixed and upper body
controlled by IK, and an environment with the lower body controlled by a policy and the upper body controlled by IK.
0.45.15 (2025-09-05)
~~~~~~~~~~~~~~~~~~~~
......@@ -5048,8 +5134,7 @@ Added
~~~~~~~~~~~~~~~~~~
* Added the :class:`isaaclab.app.AppLauncher` class to allow controlled instantiation of
the `SimulationApp <https://docs.omniverse.nvidia.com/py/isaacsim/source/isaacsim.simulation_app/docs/index.html>`_
and extension loading for remote deployment and ROS bridges.
the SimulationApp and extension loading for remote deployment and ROS bridges.
Changed
^^^^^^^
......
source/isaaclab/isaaclab/assets/surface_gripper/surface_gripper.py
View file @ 454c3d2f
......@@ -162,9 +162,9 @@ class SurfaceGripper(AssetBase):
This function is called every simulation step.
The data fetched from the gripper view is a list of strings containing 3 possible states:
- "Open"
- "Closing"
- "Closed"
- "Open" --> 0
- "Closing" --> 1
- "Closed" --> 2
To make this more neural network friendly, we convert the list of strings to a list of floats:
- "Open" --> -1.0
......@@ -175,11 +175,8 @@ class SurfaceGripper(AssetBase):
We need to do this conversion for every single step of the simulation because the gripper can lose contact
with the object if some conditions are met: such as if a large force is applied to the gripped object.
"""
state_list: list[str] = self._gripper_view.get_surface_gripper_status()
state_list_as_int: list[float] = [
-1.0 if state == "Open" else 1.0 if state == "Closed" else 0.0 for state in state_list
]
self._gripper_state = torch.tensor(state_list_as_int, dtype=torch.float32, device=self._device)
state_list: list[int] = self._gripper_view.get_surface_gripper_status()
self._gripper_state = torch.tensor(state_list, dtype=torch.float32, device=self._device) - 1.0
def write_data_to_sim(self) -> None:
"""Write the gripper command to the SurfaceGripperView.
......
source/isaaclab/isaaclab/controllers/pink_ik/local_frame_task.py 0 → 100644
View file @ 454c3d2f
# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
import numpy as np
from collections.abc import Sequence
import pinocchio as pin
from pink.tasks.frame_task import FrameTask
from .pink_kinematics_configuration import PinkKinematicsConfiguration
class LocalFrameTask(FrameTask):
"""
A task that computes error in a local (custom) frame.
Inherits from FrameTask but overrides compute_error.
"""
def __init__(
self,
frame: str,
base_link_frame_name: str,
position_cost: float | Sequence[float],
orientation_cost: float | Sequence[float],
lm_damping: float = 0.0,
gain: float = 1.0,
):
"""
Initialize the LocalFrameTask with configuration.
This task computes pose errors in a local (custom) frame rather than the world frame,
allowing for more flexible control strategies where the reference frame can be
specified independently.
Args:
frame: Name of the frame to control (end-effector or target frame).
base_link_frame_name: Name of the base link frame used as reference frame
for computing transforms and errors.
position_cost: Cost weight(s) for position error. Can be a single float
for uniform weighting or a sequence of 3 floats for per-axis weighting.
orientation_cost: Cost weight(s) for orientation error. Can be a single float
for uniform weighting or a sequence of 3 floats for per-axis weighting.
lm_damping: Levenberg-Marquardt damping factor for numerical stability.
Defaults to 0.0 (no damping).
gain: Task gain factor that scales the overall task contribution.
Defaults to 1.0.
"""
super().__init__(frame, position_cost, orientation_cost, lm_damping, gain)
self.base_link_frame_name = base_link_frame_name
self.transform_target_to_base = None
def set_target(self, transform_target_to_base: pin.SE3) -> None:
"""Set task target pose in the world frame.
Args:
transform_target_to_world: Transform from the task target frame to
the world frame.
"""
self.transform_target_to_base = transform_target_to_base.copy()
def set_target_from_configuration(self, configuration: PinkKinematicsConfiguration) -> None:
"""Set task target pose from a robot configuration.
Args:
configuration: Robot configuration.
"""
if not isinstance(configuration, PinkKinematicsConfiguration):
raise ValueError("configuration must be a PinkKinematicsConfiguration")
self.set_target(configuration.get_transform(self.frame, self.base_link_frame_name))
def compute_error(self, configuration: PinkKinematicsConfiguration) -> np.ndarray:
"""
Compute the error between current and target pose in a local frame.
"""
if not isinstance(configuration, PinkKinematicsConfiguration):
raise ValueError("configuration must be a PinkKinematicsConfiguration")
if self.transform_target_to_base is None:
raise ValueError(f"no target set for frame '{self.frame}'")
transform_frame_to_base = configuration.get_transform(self.frame, self.base_link_frame_name)
transform_target_to_frame = transform_frame_to_base.actInv(self.transform_target_to_base)
error_in_frame: np.ndarray = pin.log(transform_target_to_frame).vector
return error_in_frame
def compute_jacobian(self, configuration: PinkKinematicsConfiguration) -> np.ndarray:
r"""Compute the frame task Jacobian.
The task Jacobian :math:`J(q) \in \mathbb{R}^{6 \times n_v}` is the
derivative of the task error :math:`e(q) \in \mathbb{R}^6` with respect
to the configuration :math:`q`. The formula for the frame task is:
.. math::
J(q) = -\text{Jlog}_6(T_{tb}) {}_b J_{0b}(q)
The derivation of the formula for this Jacobian is detailed in
[Caron2023]_. See also
:func:`pink.tasks.task.Task.compute_jacobian` for more context on task
Jacobians.
Args:
configuration: Robot configuration :math:`q`.
Returns:
Jacobian matrix :math:`J`, expressed locally in the frame.
"""
if self.transform_target_to_base is None:
raise Exception(f"no target set for frame '{self.frame}'")
transform_frame_to_base = configuration.get_transform(self.frame, self.base_link_frame_name)
transform_frame_to_target = self.transform_target_to_base.actInv(transform_frame_to_base)
jacobian_in_frame = configuration.get_frame_jacobian(self.frame)
J = -pin.Jlog6(transform_frame_to_target) @ jacobian_in_frame
return J
source/isaaclab/isaaclab/controllers/pink_ik/pink_ik_cfg.py
View file @ 454c3d2f
......@@ -46,6 +46,10 @@ class PinkIKControllerCfg:
"""
joint_names: list[str] | None = None
"""A list of joint names in the USD asset controlled by the Pink IK controller. This is required because the joint naming conventions differ between USD and URDF files.
This value is currently designed to be automatically populated by the action term in a manager based environment."""
all_joint_names: list[str] | None = None
"""A list of joint names in the USD asset. This is required because the joint naming conventions differ between USD and URDF files.
This value is currently designed to be automatically populated by the action term in a manager based environment."""
......
source/isaaclab/isaaclab/controllers/pink_ik/pink_kinematics_configuration.py 0 → 100644
View file @ 454c3d2f
# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
from __future__ import annotations
import numpy as np
import pinocchio as pin
from pink.configuration import Configuration
from pink.exceptions import FrameNotFound
from pinocchio.robot_wrapper import RobotWrapper
class PinkKinematicsConfiguration(Configuration):
"""
A configuration class that maintains both a "controlled" (reduced) model and a "full" model.
This class extends the standard Pink Configuration to allow for selective joint control:
- The "controlled" model/data/q represent the subset of joints being actively controlled (e.g., a kinematic chain or arm).
- The "full" model/data/q represent the complete robot, including all joints.
This is useful for scenarios where only a subset of joints are being optimized or controlled, but full-model kinematics
(e.g., for collision checking, full-body Jacobians, or visualization) are still required.
The class ensures that both models are kept up to date, and provides methods to update both the controlled and full
configurations as needed.
"""
def __init__(
self,
controlled_joint_names: list[str],
urdf_path: str,
mesh_path: str | None = None,
copy_data: bool = True,
forward_kinematics: bool = True,
):
"""
Initialize PinkKinematicsConfiguration.
Args:
urdf_path (str): Path to the robot URDF file.
mesh_path (str): Path to the mesh files for the robot.
controlled_joint_names (list[str]): List of joint names to be actively controlled.
copy_data (bool, optional): If True, work on an internal copy of the input data. Defaults to True.
forward_kinematics (bool, optional): If True, compute forward kinematics from the configuration vector. Defaults to True.
This constructor initializes the PinkKinematicsConfiguration, which maintains both a "controlled" (reduced) model and a "full" model.
The controlled model/data/q represent the subset of joints being actively controlled, while the full model/data/q represent the complete robot.
This is useful for scenarios where only a subset of joints are being optimized or controlled, but full-model kinematics are still required.
"""
self._controlled_joint_names = controlled_joint_names
# Build robot model with all joints
if mesh_path:
self.robot_wrapper = RobotWrapper.BuildFromURDF(urdf_path, mesh_path)
else:
self.robot_wrapper = RobotWrapper.BuildFromURDF(urdf_path)
self.full_model = self.robot_wrapper.model
self.full_data = self.robot_wrapper.data
self.full_q = self.robot_wrapper.q0
# import pdb; pdb.set_trace()
self._all_joint_names = self.full_model.names.tolist()[1:]
# controlled_joint_indices: indices in all_joint_names for joints that are in controlled_joint_names, preserving all_joint_names order
self._controlled_joint_indices = [
idx for idx, joint_name in enumerate(self._all_joint_names) if joint_name in self._controlled_joint_names
]
# Build the reduced model with only the controlled joints
joints_to_lock = []
for joint_name in self._all_joint_names:
if joint_name not in self._controlled_joint_names:
joints_to_lock.append(self.full_model.getJointId(joint_name))
if len(joints_to_lock) == 0:
# No joints to lock, controlled model is the same as full model
self.controlled_model = self.full_model
self.controlled_data = self.full_data
self.controlled_q = self.full_q
else:
self.controlled_model = pin.buildReducedModel(self.full_model, joints_to_lock, self.full_q)
self.controlled_data = self.controlled_model.createData()
self.controlled_q = self.full_q[self._controlled_joint_indices]
# Pink will should only have the controlled model
super().__init__(self.controlled_model, self.controlled_data, self.controlled_q, copy_data, forward_kinematics)
def update(self, q: np.ndarray | None = None) -> None:
"""Update configuration to a new vector.
Calling this function runs forward kinematics and computes
collision-pair distances, if applicable.
Args:
q: New configuration vector.
"""
if q is not None and len(q) != len(self._all_joint_names):
raise ValueError("q must have the same length as the number of joints in the model")
if q is not None:
super().update(q[self._controlled_joint_indices])
q_readonly = q.copy()
q_readonly.setflags(write=False)
self.full_q = q_readonly
pin.computeJointJacobians(self.full_model, self.full_data, q)
pin.updateFramePlacements(self.full_model, self.full_data)
else:
super().update()
pin.computeJointJacobians(self.full_model, self.full_data, self.full_q)
pin.updateFramePlacements(self.full_model, self.full_data)
def get_frame_jacobian(self, frame: str) -> np.ndarray:
r"""Compute the Jacobian matrix of a frame velocity.
Denoting our frame by :math:`B` and the world frame by :math:`W`, the
Jacobian matrix :math:`{}_B J_{WB}` is related to the body velocity
:math:`{}_B v_{WB}` by:
.. math::
{}_B v_{WB} = {}_B J_{WB} \dot{q}
Args:
frame: Name of the frame, typically a link name from the URDF.
Returns:
Jacobian :math:`{}_B J_{WB}` of the frame.
When the robot model includes a floating base
(pin.JointModelFreeFlyer), the configuration vector :math:`q` consists
of:
- ``q[0:3]``: position in [m] of the floating base in the inertial
frame, formatted as :math:`[p_x, p_y, p_z]`.
- ``q[3:7]``: unit quaternion for the orientation of the floating base
in the inertial frame, formatted as :math:`[q_x, q_y, q_z, q_w]`.
- ``q[7:]``: joint angles in [rad].
"""
if not self.full_model.existFrame(frame):
raise FrameNotFound(frame, self.full_model.frames)
frame_id = self.full_model.getFrameId(frame)
J: np.ndarray = pin.getFrameJacobian(self.full_model, self.full_data, frame_id, pin.ReferenceFrame.LOCAL)
return J[:, self._controlled_joint_indices]
def get_transform_frame_to_world(self, frame: str) -> pin.SE3:
"""Get the pose of a frame in the current configuration.
We override this method from the super class to solve the issue that in the default
Pink implementation, the frame placements do not take into account the non-controlled joints
being not at initial pose (which is a bad assumption when they are controlled by other controllers like a lower body controller).
Args:
frame: Name of a frame, typically a link name from the URDF.
Returns:
Current transform from the given frame to the world frame.
Raises:
FrameNotFound: if the frame name is not found in the robot model.
"""
frame_id = self.full_model.getFrameId(frame)
try:
return self.full_data.oMf[frame_id].copy()
except IndexError as index_error:
raise FrameNotFound(frame, self.full_model.frames) from index_error
def check_limits(self, tol: float = 1e-6, safety_break: bool = True) -> None:
"""Check if limits are violated only if safety_break is enabled"""
if safety_break:
super().check_limits(tol, safety_break)
@property
def controlled_joint_names_pinocchio_order(self) -> list[str]:
"""Get the names of the controlled joints in the order of the pinocchio model."""
return [self._all_joint_names[i] for i in self._controlled_joint_indices]
@property
def all_joint_names_pinocchio_order(self) -> list[str]:
"""Get the names of all joints in the order of the pinocchio model."""
return self._all_joint_names
source/isaaclab/isaaclab/controllers/utils.py
View file @ 454c3d2f
......@@ -9,6 +9,7 @@ This module provides utility functions to help with controller implementations.
"""
import os
import re
from isaacsim.core.utils.extensions import enable_extension
......@@ -98,3 +99,38 @@ def change_revolute_to_fixed(urdf_path: str, fixed_joints: list[str], verbose: b
with open(urdf_path, "w") as file:
file.write(content)
def change_revolute_to_fixed_regex(urdf_path: str, fixed_joints: list[str], verbose: bool = False):
"""Change revolute joints to fixed joints in a URDF file.
This function modifies a URDF file by changing specified revolute joints to fixed joints.
This is useful when you want to disable certain joints in a robot model.
Args:
urdf_path: Path to the URDF file to modify.
fixed_joints: List of regular expressions matching joint names to convert from revolute to fixed.
verbose: Whether to print information about the changes being made.
"""
with open(urdf_path) as file:
content = file.read()
# Find all revolute joints in the URDF
revolute_joints = re.findall(r'<joint name="([^"]+)" type="revolute">', content)
for joint in revolute_joints:
# Check if this joint matches any of the fixed joint patterns
should_fix = any(re.match(pattern, joint) for pattern in fixed_joints)
if should_fix:
old_str = f'<joint name="{joint}" type="revolute">'
new_str = f'<joint name="{joint}" type="fixed">'
if verbose:
omni.log.warn(f"Replacing {joint} with fixed joint")
omni.log.warn(old_str)
omni.log.warn(new_str)
content = content.replace(old_str, new_str)
with open(urdf_path, "w") as file:
file.write(content)
source/isaaclab/isaaclab/devices/keyboard/se2_keyboard.py
View file @ 454c3d2f
......@@ -82,7 +82,7 @@ class Se2Keyboard(DeviceBase):
def __del__(self):
"""Release the keyboard interface."""
self._input.unsubscribe_from_keyboard_events(self._keyboard, self._keyboard_sub)
self._input.unsubscribe_to_keyboard_events(self._keyboard, self._keyboard_sub)
self._keyboard_sub = None
def __str__(self) -> str:
......
source/isaaclab/isaaclab/devices/keyboard/se3_keyboard.py
View file @ 454c3d2f
......@@ -90,7 +90,7 @@ class Se3Keyboard(DeviceBase):
def __del__(self):
"""Release the keyboard interface."""
self._input.unsubscribe_from_keyboard_events(self._keyboard, self._keyboard_sub)
self._input.unsubscribe_to_keyboard_events(self._keyboard, self._keyboard_sub)
self._keyboard_sub = None
def __str__(self) -> str:
......
source/isaaclab/isaaclab/devices/openxr/manus_vive_utils.py
View file @ 454c3d2f
......@@ -20,39 +20,39 @@ from pxr import Gf
# Mapping from Manus joint index (0-24) to joint name. Palm (25) is calculated from middle metacarpal and proximal.
HAND_JOINT_MAP = {
# Palm
25: "palm",
# Wrist
0: "wrist",
# Thumb
21: "thumb_metacarpal",
22: "thumb_proximal",
23: "thumb_distal",
24: "thumb_tip",
1: "thumb_metacarpal",
2: "thumb_proximal",
3: "thumb_distal",
4: "thumb_tip",
# Index
1: "index_metacarpal",
2: "index_proximal",
3: "index_intermediate",
4: "index_distal",
5: "index_tip",
5: "index_metacarpal",
6: "index_proximal",
7: "index_intermediate",
8: "index_distal",
9: "index_tip",
# Middle
6: "middle_metacarpal",
7: "middle_proximal",
8: "middle_intermediate",
9: "middle_distal",
10: "middle_tip",
10: "middle_metacarpal",
11: "middle_proximal",
12: "middle_intermediate",
13: "middle_distal",
14: "middle_tip",
# Ring
11: "ring_metacarpal",
12: "ring_proximal",
13: "ring_intermediate",
14: "ring_distal",
15: "ring_tip",
15: "ring_metacarpal",
16: "ring_proximal",
17: "ring_intermediate",
18: "ring_distal",
19: "ring_tip",
# Little
16: "little_metacarpal",
17: "little_proximal",
18: "little_intermediate",
19: "little_distal",
20: "little_tip",
20: "little_metacarpal",
21: "little_proximal",
22: "little_intermediate",
23: "little_distal",
24: "little_tip",
# Palm
25: "palm",
}
......
source/isaaclab/isaaclab/devices/openxr/retargeters/__init__.py
View file @ 454c3d2f
......@@ -5,6 +5,12 @@
"""Retargeters for mapping input device data to robot commands."""
from .humanoid.fourier.gr1t2_retargeter import GR1T2Retargeter, GR1T2RetargeterCfg
from .humanoid.unitree.g1_lower_body_standing import G1LowerBodyStandingRetargeter, G1LowerBodyStandingRetargeterCfg
from .humanoid.unitree.inspire.g1_upper_body_retargeter import UnitreeG1Retargeter, UnitreeG1RetargeterCfg
from .humanoid.unitree.trihand.g1_upper_body_retargeter import (
G1TriHandUpperBodyRetargeter,
G1TriHandUpperBodyRetargeterCfg,
)
from .manipulator.gripper_retargeter import GripperRetargeter, GripperRetargeterCfg
from .manipulator.se3_abs_retargeter import Se3AbsRetargeter, Se3AbsRetargeterCfg
from .manipulator.se3_rel_retargeter import Se3RelRetargeter, Se3RelRetargeterCfg
source/isaaclab/isaaclab/devices/openxr/retargeters/humanoid/fourier/data/configs/dex-retargeting/fourier_hand_left_dexpilot.yml
View file @ 454c3d2f
......@@ -11,7 +11,7 @@ retargeting:
- GR1T2_fourier_hand_6dof_L_ring_intermediate_link
- GR1T2_fourier_hand_6dof_L_pinky_intermediate_link
low_pass_alpha: 0.2
scaling_factor: 1.0
scaling_factor: 1.2
target_joint_names:
- L_index_proximal_joint
- L_middle_proximal_joint
......
source/isaaclab/isaaclab/devices/openxr/retargeters/humanoid/fourier/data/configs/dex-retargeting/fourier_hand_right_dexpilot.yml
View file @ 454c3d2f
......@@ -11,7 +11,7 @@ retargeting:
- GR1T2_fourier_hand_6dof_R_ring_intermediate_link
- GR1T2_fourier_hand_6dof_R_pinky_intermediate_link
low_pass_alpha: 0.2
scaling_factor: 1.0
scaling_factor: 1.2
target_joint_names:
- R_index_proximal_joint
- R_middle_proximal_joint
......
source/isaaclab/isaaclab/devices/openxr/retargeters/humanoid/unitree/g1_lower_body_standing.py 0 → 100644
View file @ 454c3d2f
# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
import torch
from dataclasses import dataclass
from isaaclab.devices.retargeter_base import RetargeterBase, RetargeterCfg
@dataclass
class G1LowerBodyStandingRetargeterCfg(RetargeterCfg):
"""Configuration for the G1 lower body standing retargeter."""
hip_height: float = 0.72
"""Height of the G1 robot hip in meters. The value is a fixed height suitable for G1 to do tabletop manipulation."""
class G1LowerBodyStandingRetargeter(RetargeterBase):
"""Provides lower body standing commands for the G1 robot."""
def __init__(self, cfg: G1LowerBodyStandingRetargeterCfg):
"""Initialize the retargeter."""
self.cfg = cfg
def retarget(self, data: dict) -> torch.Tensor:
return torch.tensor([0.0, 0.0, 0.0, self.cfg.hip_height], device=self.cfg.sim_device)
source/isaaclab/isaaclab/devices/openxr/retargeters/humanoid/unitree/inspire/data/configs/dex-retargeting/unitree_hand_left_dexpilot.yml 0 → 100644
View file @ 454c3d2f
# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
retargeting:
finger_tip_link_names:
- L_thumb_tip
- L_index_tip
- L_middle_tip
- L_ring_tip
- L_pinky_tip
low_pass_alpha: 0.2
scaling_factor: 1.2
target_joint_names:
- L_thumb_proximal_yaw_joint
- L_thumb_proximal_pitch_joint
- L_index_proximal_joint
- L_middle_proximal_joint
- L_ring_proximal_joint
- L_pinky_proximal_joint
type: DexPilot
urdf_path: /tmp/retarget_inspire_white_left_hand.urdf
wrist_link_name: L_hand_base_link
source/isaaclab/isaaclab/devices/openxr/retargeters/humanoid/unitree/inspire/data/configs/dex-retargeting/unitree_hand_right_dexpilot.yml 0 → 100644
View file @ 454c3d2f
# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
retargeting:
finger_tip_link_names:
- R_thumb_tip
- R_index_tip
- R_middle_tip
- R_ring_tip
- R_pinky_tip
low_pass_alpha: 0.2
scaling_factor: 1.2
target_joint_names:
- R_thumb_proximal_yaw_joint
- R_thumb_proximal_pitch_joint
- R_index_proximal_joint
- R_middle_proximal_joint
- R_ring_proximal_joint
- R_pinky_proximal_joint
type: DexPilot
urdf_path: /tmp/retarget_inspire_white_right_hand.urdf
wrist_link_name: R_hand_base_link
source/isaaclab/isaaclab/devices/openxr/retargeters/humanoid/unitree/inspire/g1_dex_retargeting_utils.py 0 → 100644
View file @ 454c3d2f
# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
import numpy as np
import os
import torch
import yaml
from scipy.spatial.transform import Rotation as R
import omni.log
from dex_retargeting.retargeting_config import RetargetingConfig
from isaaclab.utils.assets import ISAACLAB_NUCLEUS_DIR, retrieve_file_path
# The index to map the OpenXR hand joints to the hand joints used
# in Dex-retargeting.
_HAND_JOINTS_INDEX = [1, 2, 3, 4, 5, 7, 8, 9, 10, 12, 13, 14, 15, 17, 18, 19, 20, 22, 23, 24, 25]
# The transformation matrices to convert hand pose to canonical view.
_OPERATOR2MANO_RIGHT = np.array([
[0, -1, 0],
[-1, 0, 0],
[0, 0, -1],
])
_OPERATOR2MANO_LEFT = np.array([
[0, -1, 0],
[-1, 0, 0],
[0, 0, -1],
])
_LEFT_HAND_JOINT_NAMES = [
"L_thumb_proximal_yaw_joint",
"L_thumb_proximal_pitch_joint",
"L_thumb_intermediate_joint",
"L_thumb_distal_joint",
"L_index_proximal_joint",
"L_index_intermediate_joint",
"L_middle_proximal_joint",
"L_middle_intermediate_joint",
"L_ring_proximal_joint",
"L_ring_intermediate_joint",
"L_pinky_proximal_joint",
"L_pinky_intermediate_joint",
]
_RIGHT_HAND_JOINT_NAMES = [
"R_thumb_proximal_yaw_joint",
"R_thumb_proximal_pitch_joint",
"R_thumb_intermediate_joint",
"R_thumb_distal_joint",
"R_index_proximal_joint",
"R_index_intermediate_joint",
"R_middle_proximal_joint",
"R_middle_intermediate_joint",
"R_ring_proximal_joint",
"R_ring_intermediate_joint",
"R_pinky_proximal_joint",
"R_pinky_intermediate_joint",
]
class UnitreeG1DexRetargeting:
"""A class for hand retargeting with GR1Fourier.
Handles retargeting of OpenXRhand tracking data to GR1T2 robot hand joint angles.
"""
def __init__(
self,
hand_joint_names: list[str],
right_hand_config_filename: str = "unitree_hand_right_dexpilot.yml",
left_hand_config_filename: str = "unitree_hand_left_dexpilot.yml",
left_hand_urdf_path: str = f"{ISAACLAB_NUCLEUS_DIR}/Mimic/G1_inspire_assets/retarget_inspire_white_left_hand.urdf",
right_hand_urdf_path: str = f"{ISAACLAB_NUCLEUS_DIR}/Mimic/G1_inspire_assets/retarget_inspire_white_right_hand.urdf",
):
"""Initialize the hand retargeting.
Args:
hand_joint_names: Names of hand joints in the robot model
right_hand_config_filename: Config file for right hand retargeting
left_hand_config_filename: Config file for left hand retargeting
"""
data_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), "data/"))
config_dir = os.path.join(data_dir, "configs/dex-retargeting")
# Download urdf files from aws
local_left_urdf_path = retrieve_file_path(left_hand_urdf_path, force_download=True)
local_right_urdf_path = retrieve_file_path(right_hand_urdf_path, force_download=True)
left_config_path = os.path.join(config_dir, left_hand_config_filename)
right_config_path = os.path.join(config_dir, right_hand_config_filename)
# Update the YAML files with the correct URDF paths
self._update_yaml_with_urdf_path(left_config_path, local_left_urdf_path)
self._update_yaml_with_urdf_path(right_config_path, local_right_urdf_path)
self._dex_left_hand = RetargetingConfig.load_from_file(left_config_path).build()
self._dex_right_hand = RetargetingConfig.load_from_file(right_config_path).build()
self.left_dof_names = self._dex_left_hand.optimizer.robot.dof_joint_names
self.right_dof_names = self._dex_right_hand.optimizer.robot.dof_joint_names
self.dof_names = self.left_dof_names + self.right_dof_names
self.isaac_lab_hand_joint_names = hand_joint_names
omni.log.info("[UnitreeG1DexRetargeter] init done.")
def _update_yaml_with_urdf_path(self, yaml_path: str, urdf_path: str):
"""Update YAML file with the correct URDF path.
Args:
yaml_path: Path to the YAML configuration file
urdf_path: Path to the URDF file to use
"""
try:
# Read the YAML file
with open(yaml_path) as file:
config = yaml.safe_load(file)
# Update the URDF path in the configuration
if "retargeting" in config:
config["retargeting"]["urdf_path"] = urdf_path
omni.log.info(f"Updated URDF path in {yaml_path} to {urdf_path}")
else:
omni.log.warn(f"Unable to find 'retargeting' section in {yaml_path}")
# Write the updated configuration back to the file
with open(yaml_path, "w") as file:
yaml.dump(config, file)
except Exception as e:
omni.log.error(f"Error updating YAML file {yaml_path}: {e}")
def convert_hand_joints(self, hand_poses: dict[str, np.ndarray], operator2mano: np.ndarray) -> np.ndarray:
"""Prepares the hand joints data for retargeting.
Args:
hand_poses: Dictionary containing hand pose data with joint positions and rotations
operator2mano: Transformation matrix to convert from operator to MANO frame
Returns:
Joint positions with shape (21, 3)
"""
joint_position = np.zeros((21, 3))
hand_joints = list(hand_poses.values())
for i in range(len(_HAND_JOINTS_INDEX)):
joint = hand_joints[_HAND_JOINTS_INDEX[i]]
joint_position[i] = joint[:3]
# Convert hand pose to the canonical frame.
joint_position = joint_position - joint_position[0:1, :]
xr_wrist_quat = hand_poses.get("wrist")[3:]
# OpenXR hand uses w,x,y,z order for quaternions but scipy uses x,y,z,w order
wrist_rot = R.from_quat([xr_wrist_quat[1], xr_wrist_quat[2], xr_wrist_quat[3], xr_wrist_quat[0]]).as_matrix()
return joint_position @ wrist_rot @ operator2mano
def compute_ref_value(self, joint_position: np.ndarray, indices: np.ndarray, retargeting_type: str) -> np.ndarray:
"""Computes reference value for retargeting.
Args:
joint_position: Joint positions array
indices: Target link indices
retargeting_type: Type of retargeting ("POSITION" or other)
Returns:
Reference value in cartesian space
"""
if retargeting_type == "POSITION":
return joint_position[indices, :]
else:
origin_indices = indices[0, :]
task_indices = indices[1, :]
ref_value = joint_position[task_indices, :] - joint_position[origin_indices, :]
return ref_value
def compute_one_hand(
self, hand_joints: dict[str, np.ndarray], retargeting: RetargetingConfig, operator2mano: np.ndarray
) -> np.ndarray:
"""Computes retargeted joint angles for one hand.
Args:
hand_joints: Dictionary containing hand joint data
retargeting: Retargeting configuration object
operator2mano: Transformation matrix from operator to MANO frame
Returns:
Retargeted joint angles
"""
joint_pos = self.convert_hand_joints(hand_joints, operator2mano)
ref_value = self.compute_ref_value(
joint_pos,
indices=retargeting.optimizer.target_link_human_indices,
retargeting_type=retargeting.optimizer.retargeting_type,
)
# Enable gradient calculation and inference mode in case some other script has disabled it
# This is necessary for the retargeting to work since it uses gradient features that
# are not available in inference mode
with torch.enable_grad():
with torch.inference_mode(False):
return retargeting.retarget(ref_value)
def get_joint_names(self) -> list[str]:
"""Returns list of all joint names."""
return self.dof_names
def get_left_joint_names(self) -> list[str]:
"""Returns list of left hand joint names."""
return self.left_dof_names
def get_right_joint_names(self) -> list[str]:
"""Returns list of right hand joint names."""
return self.right_dof_names
def get_hand_indices(self, robot) -> np.ndarray:
"""Gets indices of hand joints in robot's DOF array.
Args:
robot: Robot object containing DOF information
Returns:
Array of joint indices
"""
return np.array([robot.dof_names.index(name) for name in self.dof_names], dtype=np.int64)
def compute_left(self, left_hand_poses: dict[str, np.ndarray]) -> np.ndarray:
"""Computes retargeted joints for left hand.
Args:
left_hand_poses: Dictionary of left hand joint poses
Returns:
Retargeted joint angles for left hand
"""
if left_hand_poses is not None:
left_hand_q = self.compute_one_hand(left_hand_poses, self._dex_left_hand, _OPERATOR2MANO_LEFT)
else:
left_hand_q = np.zeros(len(_LEFT_HAND_JOINT_NAMES))
return left_hand_q
def compute_right(self, right_hand_poses: dict[str, np.ndarray]) -> np.ndarray:
"""Computes retargeted joints for right hand.
Args:
right_hand_poses: Dictionary of right hand joint poses
Returns:
Retargeted joint angles for right hand
"""
if right_hand_poses is not None:
right_hand_q = self.compute_one_hand(right_hand_poses, self._dex_right_hand, _OPERATOR2MANO_RIGHT)
else:
right_hand_q = np.zeros(len(_RIGHT_HAND_JOINT_NAMES))
return right_hand_q
source/isaaclab/isaaclab/devices/openxr/retargeters/humanoid/unitree/inspire/g1_upper_body_retargeter.py 0 → 100644
View file @ 454c3d2f
# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
import contextlib
import numpy as np
import torch
from dataclasses import dataclass
import isaaclab.sim as sim_utils
import isaaclab.utils.math as PoseUtils
from isaaclab.devices import OpenXRDevice
from isaaclab.devices.retargeter_base import RetargeterBase, RetargeterCfg
from isaaclab.markers import VisualizationMarkers, VisualizationMarkersCfg
# This import exception is suppressed because g1_dex_retargeting_utils depends on pinocchio which is not available on windows
with contextlib.suppress(Exception):
from .g1_dex_retargeting_utils import UnitreeG1DexRetargeting
@dataclass
class UnitreeG1RetargeterCfg(RetargeterCfg):
"""Configuration for the UnitreeG1 retargeter."""
enable_visualization: bool = False
num_open_xr_hand_joints: int = 100
hand_joint_names: list[str] | None = None # List of robot hand joint names
class UnitreeG1Retargeter(RetargeterBase):
"""Retargets OpenXR hand tracking data to GR1T2 hand end-effector commands.
This retargeter maps hand tracking data from OpenXR to joint commands for the GR1T2 robot's hands.
It handles both left and right hands, converting poses of the hands in OpenXR format joint angles for the GR1T2 robot's hands.
"""
def __init__(
self,
cfg: UnitreeG1RetargeterCfg,
):
"""Initialize the UnitreeG1 hand retargeter.
Args:
enable_visualization: If True, visualize tracked hand joints
num_open_xr_hand_joints: Number of joints tracked by OpenXR
device: PyTorch device for computations
hand_joint_names: List of robot hand joint names
"""
self._hand_joint_names = cfg.hand_joint_names
self._hands_controller = UnitreeG1DexRetargeting(self._hand_joint_names)
# Initialize visualization if enabled
self._enable_visualization = cfg.enable_visualization
self._num_open_xr_hand_joints = cfg.num_open_xr_hand_joints
self._sim_device = cfg.sim_device
if self._enable_visualization:
marker_cfg = VisualizationMarkersCfg(
prim_path="/Visuals/markers",
markers={
"joint": sim_utils.SphereCfg(
radius=0.005,
visual_material=sim_utils.PreviewSurfaceCfg(diffuse_color=(1.0, 0.0, 0.0)),
),
},
)
self._markers = VisualizationMarkers(marker_cfg)
def retarget(self, data: dict) -> torch.Tensor:
"""Convert hand joint poses to robot end-effector commands.
Args:
data: Dictionary mapping tracking targets to joint data dictionaries.
Returns:
tuple containing:
Left wrist pose
Right wrist pose in USD frame
Retargeted hand joint angles
"""
# Access the left and right hand data using the enum key
left_hand_poses = data[OpenXRDevice.TrackingTarget.HAND_LEFT]
right_hand_poses = data[OpenXRDevice.TrackingTarget.HAND_RIGHT]
left_wrist = left_hand_poses.get("wrist")
right_wrist = right_hand_poses.get("wrist")
if self._enable_visualization:
joints_position = np.zeros((self._num_open_xr_hand_joints, 3))
joints_position[::2] = np.array([pose[:3] for pose in left_hand_poses.values()])
joints_position[1::2] = np.array([pose[:3] for pose in right_hand_poses.values()])
self._markers.visualize(translations=torch.tensor(joints_position, device=self._sim_device))
# Create array of zeros with length matching number of joint names
left_hands_pos = self._hands_controller.compute_left(left_hand_poses)
indexes = [self._hand_joint_names.index(name) for name in self._hands_controller.get_left_joint_names()]
left_retargeted_hand_joints = np.zeros(len(self._hands_controller.get_joint_names()))
left_retargeted_hand_joints[indexes] = left_hands_pos
left_hand_joints = left_retargeted_hand_joints
right_hands_pos = self._hands_controller.compute_right(right_hand_poses)
indexes = [self._hand_joint_names.index(name) for name in self._hands_controller.get_right_joint_names()]
right_retargeted_hand_joints = np.zeros(len(self._hands_controller.get_joint_names()))
right_retargeted_hand_joints[indexes] = right_hands_pos
right_hand_joints = right_retargeted_hand_joints
retargeted_hand_joints = left_hand_joints + right_hand_joints
# Convert numpy arrays to tensors and concatenate them
left_wrist_tensor = torch.tensor(
self._retarget_abs(left_wrist, True), dtype=torch.float32, device=self._sim_device
)
right_wrist_tensor = torch.tensor(
self._retarget_abs(right_wrist, False), dtype=torch.float32, device=self._sim_device
)
hand_joints_tensor = torch.tensor(retargeted_hand_joints, dtype=torch.float32, device=self._sim_device)
# Combine all tensors into a single tensor
return torch.cat([left_wrist_tensor, right_wrist_tensor, hand_joints_tensor])
def _retarget_abs(self, wrist: np.ndarray, is_left: bool) -> np.ndarray:
"""Handle absolute pose retargeting.
Args:
wrist: Wrist pose data from OpenXR.
is_left: True for the left hand, False for the right hand.
Returns:
Retargeted wrist pose in USD control frame.
"""
# Note: This was determined through trial, use the target quat and cloudXR quat,
# to estimate a most reasonable transformation matrix
wrist_pos = torch.tensor(wrist[:3], dtype=torch.float32)
wrist_quat = torch.tensor(wrist[3:], dtype=torch.float32)
if is_left:
# Corresponds to a rotation of (0, 180, 0) in euler angles (x,y,z)
combined_quat = torch.tensor([0.7071, 0, 0.7071, 0], dtype=torch.float32)
else:
# Corresponds to a rotation of (180, 0, 0) in euler angles (x,y,z)
combined_quat = torch.tensor([0, 0.7071, 0, -0.7071], dtype=torch.float32)
openxr_pose = PoseUtils.make_pose(wrist_pos, PoseUtils.matrix_from_quat(wrist_quat))
transform_pose = PoseUtils.make_pose(torch.zeros(3), PoseUtils.matrix_from_quat(combined_quat))
result_pose = PoseUtils.pose_in_A_to_pose_in_B(transform_pose, openxr_pose)
pos, rot_mat = PoseUtils.unmake_pose(result_pose)
quat = PoseUtils.quat_from_matrix(rot_mat)
return np.concatenate([pos.numpy(), quat.numpy()])
source/isaaclab/isaaclab/devices/openxr/retargeters/humanoid/unitree/trihand/data/configs/dex-retargeting/g1_hand_left_dexpilot.yml 0 → 100644
View file @ 454c3d2f
# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
retargeting:
finger_tip_link_names:
- thumb_tip
- index_tip
- middle_tip
low_pass_alpha: 0.2
scaling_factor: 1.0
target_joint_names:
- left_hand_thumb_0_joint
- left_hand_thumb_1_joint
- left_hand_thumb_2_joint
- left_hand_middle_0_joint
- left_hand_middle_1_joint
- left_hand_index_0_joint
- left_hand_index_1_joint
type: DexPilot
urdf_path: /tmp/G1_left_hand.urdf
wrist_link_name: base_link
source/isaaclab/isaaclab/devices/openxr/retargeters/humanoid/unitree/trihand/data/configs/dex-retargeting/g1_hand_right_dexpilot.yml 0 → 100644
View file @ 454c3d2f
# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
retargeting:
finger_tip_link_names:
- thumb_tip
- index_tip
- middle_tip
low_pass_alpha: 0.2
scaling_factor: 1.0
target_joint_names:
- right_hand_thumb_0_joint
- right_hand_thumb_1_joint
- right_hand_thumb_2_joint
- right_hand_middle_0_joint
- right_hand_middle_1_joint
- right_hand_index_0_joint
- right_hand_index_1_joint
type: DexPilot
urdf_path: /tmp/G1_right_hand.urdf
wrist_link_name: base_link
source/isaaclab/isaaclab/devices/openxr/retargeters/humanoid/unitree/trihand/g1_upper_body_retargeter.py 0 → 100644
View file @ 454c3d2f
# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
from __future__ import annotations
import contextlib
import numpy as np
import torch
from dataclasses import dataclass
import isaaclab.sim as sim_utils
import isaaclab.utils.math as PoseUtils
from isaaclab.devices import OpenXRDevice
from isaaclab.devices.retargeter_base import RetargeterBase, RetargeterCfg
from isaaclab.markers import VisualizationMarkers, VisualizationMarkersCfg
# This import exception is suppressed because g1_dex_retargeting_utils depends on pinocchio which is not available on windows
with contextlib.suppress(Exception):
from .g1_dex_retargeting_utils import G1TriHandDexRetargeting
@dataclass
class G1TriHandUpperBodyRetargeterCfg(RetargeterCfg):
"""Configuration for the G1UpperBody retargeter."""
enable_visualization: bool = False
num_open_xr_hand_joints: int = 100
hand_joint_names: list[str] | None = None # List of robot hand joint names
class G1TriHandUpperBodyRetargeter(RetargeterBase):
"""Retargets OpenXR data to G1 upper body commands.
This retargeter maps hand tracking data from OpenXR to wrist and hand joint commands for the G1 robot.
It handles both left and right hands, converting poses of the hands in OpenXR format to appropriate wrist poses
and joint angles for the G1 robot's upper body.
"""
def __init__(
self,
cfg: G1TriHandUpperBodyRetargeterCfg,
):
"""Initialize the G1 upper body retargeter.
Args:
cfg: Configuration for the retargeter.
"""
# Store device name for runtime retrieval
self._sim_device = cfg.sim_device
self._hand_joint_names = cfg.hand_joint_names
# Initialize the hands controller
if cfg.hand_joint_names is not None:
self._hands_controller = G1TriHandDexRetargeting(cfg.hand_joint_names)
else:
raise ValueError("hand_joint_names must be provided in configuration")
# Initialize visualization if enabled
self._enable_visualization = cfg.enable_visualization
self._num_open_xr_hand_joints = cfg.num_open_xr_hand_joints
if self._enable_visualization:
marker_cfg = VisualizationMarkersCfg(
prim_path="/Visuals/g1_hand_markers",
markers={
"joint": sim_utils.SphereCfg(
radius=0.005,
visual_material=sim_utils.PreviewSurfaceCfg(diffuse_color=(1.0, 0.0, 0.0)),
),
},
)
self._markers = VisualizationMarkers(marker_cfg)
def retarget(self, data: dict) -> torch.Tensor:
"""Convert hand joint poses to robot end-effector commands.
Args:
data: Dictionary mapping tracking targets to joint data dictionaries.
Returns:
A tensor containing the retargeted commands:
- Left wrist pose (7)
- Right wrist pose (7)
- Hand joint angles (len(hand_joint_names))
"""
# Access the left and right hand data using the enum key
left_hand_poses = data[OpenXRDevice.TrackingTarget.HAND_LEFT]
right_hand_poses = data[OpenXRDevice.TrackingTarget.HAND_RIGHT]
left_wrist = left_hand_poses.get("wrist")
right_wrist = right_hand_poses.get("wrist")
# Handle case where wrist data is not available
if left_wrist is None or right_wrist is None:
# Set to default pose if no data available.
# pos=(0,0,0), quat=(1,0,0,0) (w,x,y,z)
default_pose = np.array([0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0])
if left_wrist is None:
left_wrist = default_pose
if right_wrist is None:
right_wrist = default_pose
# Visualization if enabled
if self._enable_visualization:
joints_position = np.zeros((self._num_open_xr_hand_joints, 3))
joints_position[::2] = np.array([pose[:3] for pose in left_hand_poses.values()])
joints_position[1::2] = np.array([pose[:3] for pose in right_hand_poses.values()])
self._markers.visualize(translations=torch.tensor(joints_position, device=self._sim_device))
# Compute retargeted hand joints
left_hands_pos = self._hands_controller.compute_left(left_hand_poses)
indexes = [self._hand_joint_names.index(name) for name in self._hands_controller.get_left_joint_names()]
left_retargeted_hand_joints = np.zeros(len(self._hands_controller.get_joint_names()))
left_retargeted_hand_joints[indexes] = left_hands_pos
left_hand_joints = left_retargeted_hand_joints
right_hands_pos = self._hands_controller.compute_right(right_hand_poses)
indexes = [self._hand_joint_names.index(name) for name in self._hands_controller.get_right_joint_names()]
right_retargeted_hand_joints = np.zeros(len(self._hands_controller.get_joint_names()))
right_retargeted_hand_joints[indexes] = right_hands_pos
right_hand_joints = right_retargeted_hand_joints
retargeted_hand_joints = left_hand_joints + right_hand_joints
# Convert numpy arrays to tensors and store in command buffer
left_wrist_tensor = torch.tensor(
self._retarget_abs(left_wrist, is_left=True), dtype=torch.float32, device=self._sim_device
)
right_wrist_tensor = torch.tensor(
self._retarget_abs(right_wrist, is_left=False), dtype=torch.float32, device=self._sim_device
)
hand_joints_tensor = torch.tensor(retargeted_hand_joints, dtype=torch.float32, device=self._sim_device)
# Combine all tensors into a single tensor
return torch.cat([left_wrist_tensor, right_wrist_tensor, hand_joints_tensor])
def _retarget_abs(self, wrist: np.ndarray, is_left: bool) -> np.ndarray:
"""Handle absolute pose retargeting.
Args:
wrist: Wrist pose data from OpenXR.
is_left: True for the left hand, False for the right hand.
Returns:
Retargeted wrist pose in USD control frame.
"""
wrist_pos = torch.tensor(wrist[:3], dtype=torch.float32)
wrist_quat = torch.tensor(wrist[3:], dtype=torch.float32)
if is_left:
# Corresponds to a rotation of (0, 90, 90) in euler angles (x,y,z)
combined_quat = torch.tensor([0.7071, 0, 0.7071, 0], dtype=torch.float32)
else:
# Corresponds to a rotation of (0, -90, -90) in euler angles (x,y,z)
combined_quat = torch.tensor([0, -0.7071, 0, 0.7071], dtype=torch.float32)
openxr_pose = PoseUtils.make_pose(wrist_pos, PoseUtils.matrix_from_quat(wrist_quat))
transform_pose = PoseUtils.make_pose(torch.zeros(3), PoseUtils.matrix_from_quat(combined_quat))
result_pose = PoseUtils.pose_in_A_to_pose_in_B(transform_pose, openxr_pose)
pos, rot_mat = PoseUtils.unmake_pose(result_pose)
quat = PoseUtils.quat_from_matrix(rot_mat)
return np.concatenate([pos.numpy(), quat.numpy()])
source/isaaclab/isaaclab/devices/teleop_device_factory.py
View file @ 454c3d2f
......@@ -15,6 +15,10 @@ from isaaclab.devices import DeviceBase, DeviceCfg
from isaaclab.devices.gamepad import Se2Gamepad, Se2GamepadCfg, Se3Gamepad, Se3GamepadCfg
from isaaclab.devices.keyboard import Se2Keyboard, Se2KeyboardCfg, Se3Keyboard, Se3KeyboardCfg
from isaaclab.devices.openxr.retargeters import (
G1LowerBodyStandingRetargeter,
G1LowerBodyStandingRetargeterCfg,
G1TriHandUpperBodyRetargeter,
G1TriHandUpperBodyRetargeterCfg,
GR1T2Retargeter,
GR1T2RetargeterCfg,
GripperRetargeter,
......@@ -23,6 +27,8 @@ from isaaclab.devices.openxr.retargeters import (
Se3AbsRetargeterCfg,
Se3RelRetargeter,
Se3RelRetargeterCfg,
UnitreeG1Retargeter,
UnitreeG1RetargeterCfg,
)
from isaaclab.devices.retargeter_base import RetargeterBase, RetargeterCfg
from isaaclab.devices.spacemouse import Se2SpaceMouse, Se2SpaceMouseCfg, Se3SpaceMouse, Se3SpaceMouseCfg
......@@ -50,6 +56,9 @@ RETARGETER_MAP: dict[type[RetargeterCfg], type[RetargeterBase]] = {
Se3RelRetargeterCfg: Se3RelRetargeter,
GripperRetargeterCfg: GripperRetargeter,
GR1T2RetargeterCfg: GR1T2Retargeter,
G1TriHandUpperBodyRetargeterCfg: G1TriHandUpperBodyRetargeter,
G1LowerBodyStandingRetargeterCfg: G1LowerBodyStandingRetargeter,
UnitreeG1RetargeterCfg: UnitreeG1Retargeter,
}
......
source/isaaclab/isaaclab/envs/mdp/actions/pink_actions_cfg.py
View file @ 454c3d2f
......@@ -26,15 +26,15 @@ class PinkInverseKinematicsActionCfg(ActionTermCfg):
pink_controlled_joint_names: list[str] = MISSING
"""List of joint names or regular expression patterns that specify the joints controlled by pink IK."""
ik_urdf_fixed_joint_names: list[str] = MISSING
"""List of joint names that specify the joints to be locked in URDF."""
hand_joint_names: list[str] = MISSING
"""List of joint names or regular expression patterns that specify the joints controlled by hand retargeting."""
controller: PinkIKControllerCfg = MISSING
"""Configuration for the Pink IK controller that will be used to solve the inverse kinematics."""
enable_gravity_compensation: bool = True
"""Whether to compensate for gravity in the Pink IK controller."""
target_eef_link_names: dict[str, str] = MISSING
"""Dictionary mapping task names to controlled link names for the Pink IK controller.
......
source/isaaclab/isaaclab/sim/converters/mjcf_converter.py
View file @ 454c3d2f
......@@ -31,7 +31,7 @@ class MjcfConverter(AssetConverterBase):
From Isaac Sim 4.5 onwards, the extension name changed from ``omni.importer.mjcf`` to
``isaacsim.asset.importer.mjcf``. This converter class now uses the latest extension from Isaac Sim.
.. _isaacsim.asset.importer.mjcf: https://docs.isaacsim.omniverse.nvidia.com/latest/robot_setup/ext_isaacsim_asset_importer_mjcf.html
.. _isaacsim.asset.importer.mjcf: https://docs.isaacsim.omniverse.nvidia.com/latest/importer_exporter/ext_isaacsim_asset_importer_mjcf.html
"""
cfg: MjcfConverterCfg
......
source/isaaclab/isaaclab/sim/converters/urdf_converter.py
View file @ 454c3d2f
......@@ -34,7 +34,7 @@ class UrdfConverter(AssetConverterBase):
From Isaac Sim 4.5 onwards, the extension name changed from ``omni.importer.urdf`` to
``isaacsim.asset.importer.urdf``. This converter class now uses the latest extension from Isaac Sim.
.. _isaacsim.asset.importer.urdf: https://docs.isaacsim.omniverse.nvidia.com/latest/robot_setup/ext_isaacsim_asset_importer_urdf.html
.. _isaacsim.asset.importer.urdf: https://docs.isaacsim.omniverse.nvidia.com/latest/importer_exporter/ext_isaacsim_asset_importer_urdf.html
"""
cfg: UrdfConverterCfg
......
source/isaaclab/isaaclab/sim/simulation_cfg.py
View file @ 454c3d2f
......@@ -292,6 +292,25 @@ class RenderCfg:
This is set by the variable: ``/rtx/ambientOcclusion/enabled``.
"""
dome_light_upper_lower_strategy: Literal[0, 3, 4] | None = None
"""Selects how to sample the Dome Light. Default is 0.
For more information, refer to the `documentation`_.
.. _documentation: https://docs.omniverse.nvidia.com/materials-and-rendering/latest/rtx-renderer_common.html#dome-light
Valid values are:
* 0: **Image-Based Lighting (IBL)** - Most accurate even for high-frequency Dome Light textures.
Can introduce sampling artifacts in real-time mode.
* 3: **Limited Image-Based Lighting** - Only sampled for reflection and refraction. Fastest, but least
accurate. Good for cases where the Dome Light contributes less than other light sources.
* 4: **Approximated Image-Based Lighting** - Fast and artifacts-free sampling in real-time mode but only
works well with a low-frequency texture (e.g., a sky with no sun disc where the sun is instead a separate
Distant Light). Requires enabling Direct Lighting denoiser.
This is set by the variable: ``/rtx/domeLight/upperLowerStrategy``.
"""
carb_settings: dict[str, Any] | None = None
"""A general dictionary for users to supply all carb rendering settings with native names.
......
source/isaaclab/isaaclab/sim/simulation_context.py
View file @ 454c3d2f
......@@ -692,6 +692,7 @@ class SimulationContext(_SimulationContext):
"samples_per_pixel": "/rtx/directLighting/sampledLighting/samplesPerPixel",
"enable_shadows": "/rtx/shadows/enabled",
"enable_ambient_occlusion": "/rtx/ambientOcclusion/enabled",
"dome_light_upper_lower_strategy": "/rtx/domeLight/upperLowerStrategy",
}
not_carb_settings = ["rendering_mode", "carb_settings", "antialiasing_mode"]
......
source/isaaclab/isaaclab/sim/spawners/sensors/sensors_cfg.py
View file @ 454c3d2f
......@@ -166,7 +166,7 @@ class FisheyeCameraCfg(PinholeCameraCfg):
`camera documentation <https://docs.omniverse.nvidia.com/materials-and-rendering/latest/cameras.html#fisheye-properties>`__.
.. note::
The default values are taken from the `Replicator camera <https://docs.omniverse.nvidia.com/py/replicator/1.9.8/source/omni.replicator.core/docs/API.html#omni.replicator.core.create.camera>`__
The default values are taken from the `Replicator camera <https://docs.omniverse.nvidia.com/py/replicator/1.12.16/source/extensions/omni.replicator.core/docs/API.html#cameras>`__
function.
.. _fish-eye camera: https://en.wikipedia.org/wiki/Fisheye_lens
......
source/isaaclab/isaaclab/utils/io/__init__.py
View file @ 454c3d2f
......@@ -7,4 +7,5 @@
Submodules for files IO operations.
"""
from .torchscript import load_torchscript_model
from .yaml import dump_yaml, load_yaml
source/isaaclab/isaaclab/utils/io/torchscript.py 0 → 100644
View file @ 454c3d2f
# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
"""TorchScript I/O utilities."""
import os
import torch
def load_torchscript_model(model_path: str, device: str = "cpu") -> torch.nn.Module:
"""Load a TorchScript model from the specified path.
This function only loads TorchScript models (.pt or .pth files created with torch.jit.save).
It will not work with raw PyTorch checkpoints (.pth files created with torch.save).
Args:
model_path (str): Path to the TorchScript model file (.pt or .pth)
device (str, optional): Device to load the model on. Defaults to 'cpu'.
Returns:
torch.nn.Module: The loaded TorchScript model in evaluation mode
Raises:
FileNotFoundError: If the model file does not exist
"""
if not os.path.exists(model_path):
raise FileNotFoundError(f"TorchScript model file not found: {model_path}")
try:
model = torch.jit.load(model_path, map_location=device)
model.eval()
print(f"Successfully loaded TorchScript model from {model_path}")
return model
except Exception as e:
print(f"Error loading TorchScript model: {e}")
return None
source/isaaclab/setup.py
View file @ 454c3d2f
......@@ -35,7 +35,7 @@ INSTALL_REQUIRES = [
"einops", # needed for transformers, doesn't always auto-install
"warp-lang",
# make sure this is consistent with isaac sim version
"pillow==11.2.1",
"pillow==11.3.0",
# livestream
"starlette==0.45.3",
# testing
......@@ -46,13 +46,14 @@ INSTALL_REQUIRES = [
"flaky",
]
# Append Linux x86_64–only deps via PEP 508 markers
X64 = "platform_machine in 'x86_64,AMD64'"
# Append Linux x86_64 and ARM64 deps via PEP 508 markers
SUPPORTED_ARCHS_ARM = "platform_machine in 'x86_64,AMD64,aarch64,arm64'"
SUPPORTED_ARCHS = "platform_machine in 'x86_64,AMD64'"
INSTALL_REQUIRES += [
# required by isaaclab.isaaclab.controllers.pink_ik
f"pin-pink==3.1.0 ; platform_system == 'Linux' and ({X64})",
f"pin-pink==3.1.0 ; platform_system == 'Linux' and ({SUPPORTED_ARCHS_ARM})",
# required by isaaclab.devices.openxr.retargeters.humanoid.fourier.gr1_t2_dex_retargeting_utils
f"dex-retargeting==0.4.6 ; platform_system == 'Linux' and ({X64})",
f"dex-retargeting==0.4.6 ; platform_system == 'Linux' and ({SUPPORTED_ARCHS})",
]
PYTORCH_INDEX_URL = ["https://download.pytorch.org/whl/cu128"]
......@@ -78,6 +79,7 @@ setup(
"Programming Language :: Python :: 3.11",
"Isaac Sim :: 4.5.0",
"Isaac Sim :: 5.0.0",
"Isaac Sim :: 5.1.0",
],
zip_safe=False,
)
source/isaaclab/test/controllers/test_controller_utils.py 0 → 100644
View file @ 454c3d2f
This diff is collapsed.
source/isaaclab/test/controllers/test_ik_configs/README.md 0 → 100644
View file @ 454c3d2f
This diff is collapsed.
source/isaaclab/test/controllers/test_local_frame_task.py 0 → 100644
View file @ 454c3d2f
This diff is collapsed.
source/isaaclab/test/controllers/test_pink_ik_components.py 0 → 100644
View file @ 454c3d2f
This diff is collapsed.
source/isaaclab_assets/docs/CHANGELOG.rst
View file @ 454c3d2f
This diff is collapsed.
source/isaaclab_assets/setup.py
View file @ 454c3d2f
This diff is collapsed.
source/isaaclab_mimic/config/extension.toml
View file @ 454c3d2f
This diff is collapsed.
source/isaaclab_mimic/docs/CHANGELOG.rst
View file @ 454c3d2f
This diff is collapsed.
source/isaaclab_mimic/setup.py
View file @ 454c3d2f
This diff is collapsed.
source/isaaclab_rl/config/extension.toml
View file @ 454c3d2f
This diff is collapsed.
source/isaaclab_rl/docs/CHANGELOG.rst
View file @ 454c3d2f
This diff is collapsed.
source/isaaclab_rl/setup.py
View file @ 454c3d2f
This diff is collapsed.
source/isaaclab_tasks/config/extension.toml
View file @ 454c3d2f
This diff is collapsed.
source/isaaclab_tasks/docs/CHANGELOG.rst
View file @ 454c3d2f
This diff is collapsed.
source/isaaclab_tasks/setup.py
View file @ 454c3d2f
This diff is collapsed.
tools/template/templates/extension/setup.py
View file @ 454c3d2f
This diff is collapsed.
tools/test_settings.py
View file @ 454c3d2f
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment