[Newton] Adds policy transfer script for sim2sim transfer from Newton to physX (#3565)

# Description

This PR adds a play script to physX-based IsaacLab to make it possible
to play a Newton-based trained policy. Tested environments are
H1/G1/Anymal-D but other exisiting Newton environment should transfer as
well.


<!--
Thank you for your interest in sending a pull request. Please make sure
to check the contribution guidelines.

Link:
https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html

💡 Please try to keep PRs small and focused. Large PRs are harder to
review and merge.
-->

Please include a summary of the change and which issue is fixed. Please
also include relevant motivation and context.
List any dependencies that are required for this change.

Fixes # (issue)

<!-- As a practice, it is recommended to open an issue to have
discussions on the proposed pull request.
This makes it easier for the community to keep track of what is being
developed or added, and if a given feature
is demanded by more than one party. -->

## Type of change

<!-- As you go through the list, delete the ones that are not
applicable. -->

- Bug fix (non-breaking change which fixes an issue)
- New feature (non-breaking change which adds functionality)
- Breaking change (existing functionality will not work without user
modification)
- Documentation update

## Screenshots

Please attach before and after screenshots of the change if applicable.

<!--
Example:

| Before | After |
| ------ | ----- |
| _gif/png before_ | _gif/png after_ |

To upload images to a PR -- simply drag and drop an image while in edit
mode and it should upload the image directly. You can then paste that
source into the above before/after sections.
-->

## Checklist

- [ ] I have read and understood the [contribution
guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html)
- [ ] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [ ] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

<!--
As you go through the checklist above, you can mark something as done by
putting an x character in it

For example,
- [x] I have done this task
- [ ] I have not done this task
-->

---------
Signed-off-by: Milad Rakhsha <mrakhsha@nvidia.com>

docs/source/experimental-features/newton-physics-integration/sim-to-sim.rst

@@ -2,38 +2,40 @@
 
 Sim-to-Sim Policy Transfer
 ==========================
-This section provides examples of sim-to-sim policy transfer using the Newton backend. Sim-to-sim transfer is an essential step before real robot deployment because it verifies that policies work across different simulators. Policies that pass sim-to-sim verification are much more likely to succeed on real robots.
+This section provides examples of sim-to-sim policy transfer between PhysX and Newton backends. Sim-to-sim transfer is an essential step before real robot deployment because it verifies that policies work across different simulators. Policies that pass sim-to-sim verification are much more likely to succeed on real robots.
 
 
 Overview
 --------
 
-This guide shows how to run a PhysX-trained policy on the Newton backend. While the method works for any robot and physics engine, it has only been tested with Unitree G1, Unitree H1, and ANYmal-D robots using PhysX-trained policies.
+This guide shows how to transfer policies between PhysX and Newton backends in both directions. The main challenge is that different physics engines may parse the same robot model with different joint and link ordering.
 
-PhysX-trained policies expect joints and links in a specific order determined by how PhysX parses the robot model. However, Newton may parse the same robot with different joint and link ordering.
+Policies trained in one backend expect joints and links in a specific order determined by how that backend parses the robot model. When transferring to another backend, the joint ordering may be different, requiring remapping of observations and actions.
 
 In the future, we plan to solve this using **robot schema** that standardizes joint and link ordering across different backends.
 
-Currently, we solve this by remapping observations and actions using joint mappings defined in YAML files. These files specify joint names in both PhysX order (source) and Newton order (target). During policy execution, we use this mapping to reorder observations and actions so they work correctly with Newton.
+Currently, we solve this by remapping observations and actions using joint mappings defined in YAML files. These files specify joint names in both source and target backend orders. During policy execution, we use this mapping to reorder observations and actions so they work correctly with the target backend.
+
+The method has been tested with Unitree G1, Unitree Go2, Unitree H1, and ANYmal-D robots for both transfer directions.
 
 
 What you need
 ~~~~~~~~~~~~~
 
-- A policy checkpoint trained with PhysX (RSL-RL).
-- A joint mapping YAML for your robot under ``scripts/newton_sim2sim/mappings/``.
-- The provided player script: ``scripts/newton_sim2sim/rsl_rl_transfer.py``.
+- A policy checkpoint trained with either PhysX or Newton (RSL-RL).
+- A joint mapping YAML for your robot under ``scripts/sim2sim_transfer/config/``.
+- The provided player script: ``scripts/sim2sim_transfer/rsl_rl_transfer.py``.
 
 To add a new robot, create a YAML file with two lists where each joint name appears exactly once in both:
 
 .. code-block:: yaml
 
    # Example structure
-   source_joint_names:  # PhysX joint order
+   source_joint_names:  # Source backend joint order
      - joint_1
      - joint_2
      # ...
-   target_joint_names:  # Newton joint order
+   target_joint_names:  # Target backend joint order
      - joint_1
      - joint_2
      # ...
@@ -41,14 +43,14 @@ To add a new robot, create a YAML file with two lists where each joint name appe
 The script automatically computes the necessary mappings for locomotion tasks.
 
 
-How to run
-~~~~~~~~~~
+PhysX-to-Newton Transfer
+~~~~~~~~~~~~~~~~~~~~~~~~
 
-Use this command template to run a PhysX-trained policy with Newton:
+To run a PhysX-trained policy with the Newton backend, use this command template:
 
 .. code-block:: bash
 
-   ./isaaclab.sh -p scripts/newton_sim2sim/rsl_rl_transfer.py \
+   ./isaaclab.sh -p scripts/sim2sim_transfer/rsl_rl_transfer.py \
        --task=<TASK_ID> \
        --num_envs=32 \
        --checkpoint <PATH_TO_PHYSX_CHECKPOINT> \
@@ -60,11 +62,11 @@ Here are examples for different robots:
 
 .. code-block:: bash
 
-   ./isaaclab.sh -p scripts/newton_sim2sim/rsl_rl_transfer.py \
+   ./isaaclab.sh -p scripts/sim2sim_transfer/rsl_rl_transfer.py \
        --task=Isaac-Velocity-Flat-G1-v0 \
        --num_envs=32 \
        --checkpoint <PATH_TO_PHYSX_CHECKPOINT> \
-       --policy_transfer_file scripts/newton_sim2sim/mappings/sim2sim_g1.yaml
+       --policy_transfer_file scripts/sim2sim_transfer/config/physx_to_newton_g1.yaml
 
 
 2. Unitree H1
@@ -72,28 +74,94 @@ Here are examples for different robots:
 
 .. code-block:: bash
 
-   ./isaaclab.sh -p scripts/newton_sim2sim/rsl_rl_transfer.py \
+   ./isaaclab.sh -p scripts/sim2sim_transfer/rsl_rl_transfer.py \
        --task=Isaac-Velocity-Flat-H1-v0 \
        --num_envs=32 \
        --checkpoint <PATH_TO_PHYSX_CHECKPOINT> \
-       --policy_transfer_file scripts/newton_sim2sim/mappings/sim2sim_h1.yaml
+       --policy_transfer_file scripts/sim2sim_transfer/config/physx_to_newton_h1.yaml
+
+
+3. Unitree Go2
 
+.. code-block:: bash
 
-3. ANYmal-D
+   ./isaaclab.sh -p scripts/sim2sim_transfer/rsl_rl_transfer.py \
+       --task=Isaac-Velocity-Flat-Go2-v0 \
+       --num_envs=32 \
+       --checkpoint <PATH_TO_PHYSX_CHECKPOINT> \
+       --policy_transfer_file scripts/sim2sim_transfer/config/physx_to_newton_go2.yaml
+
+
+4. ANYmal-D
 
 
 .. code-block:: bash
 
-   ./isaaclab.sh -p scripts/newton_sim2sim/rsl_rl_transfer.py \
+   ./isaaclab.sh -p scripts/sim2sim_transfer/rsl_rl_transfer.py \
        --task=Isaac-Velocity-Flat-Anymal-D-v0 \
        --num_envs=32 \
        --checkpoint <PATH_TO_PHYSX_CHECKPOINT> \
-       --policy_transfer_file scripts/newton_sim2sim/mappings/sim2sim_anymal_d.yaml
+       --policy_transfer_file scripts/sim2sim_transfer/config/physx_to_newton_anymal_d.yaml
+
+Note that to run this, you need to checkout the Newton-based branch of IsaacLab such as ``feature/newton``.
+
+Newton-to-PhysX Transfer
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+To transfer Newton-trained policies to PhysX-based IsaacLab, use the reverse mapping files:
+
+Here are examples for different robots:
+
+1. Unitree G1
+
+.. code-block:: bash
+
+   ./isaaclab.sh -p scripts/sim2sim_transfer/rsl_rl_transfer.py \
+       --task=Isaac-Velocity-Flat-G1-v0 \
+       --num_envs=32 \
+       --checkpoint <PATH_TO_NEWTON_CHECKPOINT> \
+       --policy_transfer_file scripts/sim2sim_transfer/config/newton_to_physx_g1.yaml
+
+
+2. Unitree H1
+
+.. code-block:: bash
+
+   ./isaaclab.sh -p scripts/sim2sim_transfer/rsl_rl_transfer.py \
+       --task=Isaac-Velocity-Flat-H1-v0 \
+       --num_envs=32 \
+       --checkpoint <PATH_TO_NEWTON_CHECKPOINT> \
+       --policy_transfer_file scripts/sim2sim_transfer/config/newton_to_physx_h1.yaml
+
+
+3. Unitree Go2
+
+.. code-block:: bash
+
+   ./isaaclab.sh -p scripts/sim2sim_transfer/rsl_rl_transfer.py \
+       --task=Isaac-Velocity-Flat-Go2-v0 \
+       --num_envs=32 \
+       --checkpoint <PATH_TO_NEWTON_CHECKPOINT> \
+       --policy_transfer_file scripts/sim2sim_transfer/config/newton_to_physx_go2.yaml
+
+
+4. ANYmal-D
+
+.. code-block:: bash
+
+   ./isaaclab.sh -p scripts/sim2sim_transfer/rsl_rl_transfer.py \
+       --task=Isaac-Velocity-Flat-Anymal-D-v0 \
+       --num_envs=32 \
+       --checkpoint <PATH_TO_NEWTON_CHECKPOINT> \
+       --policy_transfer_file scripts/sim2sim_transfer/config/newton_to_physx_anymal_d.yaml
 
+The key difference is using the ``newton_to_physx_*.yaml`` mapping files instead of ``physx_to_newton_*.yaml`` files. Also note that you need to checkout a PhysX-based IsaacLab branch such as ``main``.
 
-Notes and limitations
+Notes and Limitations
 ~~~~~~~~~~~~~~~~~~~~~
 
-- This transfer method has only been tested with Unitree G1, Unitree H1, and ANYmal-D using PhysX-trained policies.
-- The observation remapping assumes a locomotion layout with base observations followed by joint observations. For different observation layouts, you'll need to modify ``scripts/newton_sim2sim/policy_mapping.py``.
+- Both transfer directions have been tested with Unitree G1, Unitree Go2, Unitree H1, and ANYmal-D robots.
+- PhysX-to-Newton transfer uses ``physx_to_newton_*.yaml`` mapping files.
+- Newton-to-PhysX transfer requires the corresponding ``newton_to_physx_*.yaml`` mapping files and the PhysX branch of IsaacLab.
+- The observation remapping assumes a locomotion layout with base observations followed by joint observations. For different observation layouts, you'll need to modify the ``get_joint_mappings`` function in ``scripts/sim2sim_transfer/rsl_rl_transfer.py``.
 - When adding new robots or backends, make sure both source and target have identical joint names, and that the YAML lists reflect how each backend orders these joints.

scripts/sim2sim_transfer/config/newton_to_physx_anymal_d.yaml

+# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
+# All rights reserved.
+#
+# SPDX-License-Identifier: BSD-3-Clause
+
+# Joint names in the source physics engine where policy is trained (Newton)
+source_joint_names:
+  - "LF_HAA"
+  - "LF_HFE"
+  - "LF_KFE"
+  - "LH_HAA"
+  - "LH_HFE"
+  - "LH_KFE"
+  - "RF_HAA"
+  - "RF_HFE"
+  - "RF_KFE"
+  - "RH_HAA"
+  - "RH_HFE"
+  - "RH_KFE"
+
+# Joint names in the target physics engine where policy is deployed (PhysX)
+target_joint_names:
+  - "LF_HAA"
+  - "LH_HAA"
+  - "RF_HAA"
+  - "RH_HAA"
+  - "LF_HFE"
+  - "LH_HFE"
+  - "RF_HFE"
+  - "RH_HFE"
+  - "LF_KFE"
+  - "LH_KFE"
+  - "RF_KFE"
+  - "RH_KFE"

scripts/sim2sim_transfer/config/newton_to_physx_g1.yaml

+# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
+# All rights reserved.
+#
+# SPDX-License-Identifier: BSD-3-Clause
+
+# Joint names in the source physics engine where policy is trained (Newton)
+source_joint_names:
+  - "left_hip_pitch_joint"
+  - "left_hip_roll_joint"
+  - "left_hip_yaw_joint"
+  - "left_knee_joint"
+  - "left_ankle_pitch_joint"
+  - "left_ankle_roll_joint"
+  - "right_hip_pitch_joint"
+  - "right_hip_roll_joint"
+  - "right_hip_yaw_joint"
+  - "right_knee_joint"
+  - "right_ankle_pitch_joint"
+  - "right_ankle_roll_joint"
+  - "torso_joint"
+  - "left_shoulder_pitch_joint"
+  - "left_shoulder_roll_joint"
+  - "left_shoulder_yaw_joint"
+  - "left_elbow_pitch_joint"
+  - "left_elbow_roll_joint"
+  - "left_five_joint"
+  - "left_six_joint"
+  - "left_three_joint"
+  - "left_four_joint"
+  - "left_zero_joint"
+  - "left_one_joint"
+  - "left_two_joint"
+  - "right_shoulder_pitch_joint"
+  - "right_shoulder_roll_joint"
+  - "right_shoulder_yaw_joint"
+  - "right_elbow_pitch_joint"
+  - "right_elbow_roll_joint"
+  - "right_five_joint"
+  - "right_six_joint"
+  - "right_three_joint"
+  - "right_four_joint"
+  - "right_zero_joint"
+  - "right_one_joint"
+  - "right_two_joint"
+
+# Joint names in the target physics engine where policy is deployed (PhysX)
+target_joint_names:
+  - "left_hip_pitch_joint"
+  - "right_hip_pitch_joint"
+  - "torso_joint"
+  - "left_hip_roll_joint"
+  - "right_hip_roll_joint"
+  - "left_shoulder_pitch_joint"
+  - "right_shoulder_pitch_joint"
+  - "left_hip_yaw_joint"
+  - "right_hip_yaw_joint"
+  - "left_shoulder_roll_joint"
+  - "right_shoulder_roll_joint"
+  - "left_knee_joint"
+  - "right_knee_joint"
+  - "left_shoulder_yaw_joint"
+  - "right_shoulder_yaw_joint"
+  - "left_ankle_pitch_joint"
+  - "right_ankle_pitch_joint"
+  - "left_elbow_pitch_joint"
+  - "right_elbow_pitch_joint"
+  - "left_ankle_roll_joint"
+  - "right_ankle_roll_joint"
+  - "left_elbow_roll_joint"
+  - "right_elbow_roll_joint"
+  - "left_five_joint"
+  - "left_three_joint"
+  - "left_zero_joint"
+  - "right_five_joint"
+  - "right_three_joint"
+  - "right_zero_joint"
+  - "left_six_joint"
+  - "left_four_joint"
+  - "left_one_joint"
+  - "right_six_joint"
+  - "right_four_joint"
+  - "right_one_joint"
+  - "left_two_joint"
+  - "right_two_joint"

scripts/sim2sim_transfer/config/newton_to_physx_go2.yaml

+# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
+# All rights reserved.
+#
+# SPDX-License-Identifier: BSD-3-Clause
+
+# Joint names in the source physics engine where policy is trained (Newton)
+source_joint_names:
+  - "FL_hip_joint"
+  - "FL_thigh_joint"
+  - "FL_calf_joint"
+  - "FR_hip_joint"
+  - "FR_thigh_joint"
+  - "FR_calf_joint"
+  - "RL_hip_joint"
+  - "RL_thigh_joint"
+  - "RL_calf_joint"
+  - "RR_hip_joint"
+  - "RR_thigh_joint"
+  - "RR_calf_joint"
+# Joint names in the target physics engine where policy is deployed (PhysX)
+target_joint_names:
+  - "FL_hip_joint"
+  - "FR_hip_joint"
+  - "RL_hip_joint"
+  - "RR_hip_joint"
+  - "FL_thigh_joint"
+  - "FR_thigh_joint"
+  - "RL_thigh_joint"
+  - "RR_thigh_joint"
+  - "FL_calf_joint"
+  - "FR_calf_joint"
+  - "RL_calf_joint"
+  - "RR_calf_joint"

scripts/sim2sim_transfer/config/newton_to_physx_h1.yaml

+# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
+# All rights reserved.
+#
+# SPDX-License-Identifier: BSD-3-Clause
+
+# Joint names in the source physics engine where policy is trained (Newton)
+source_joint_names:
+  - "left_hip_yaw"
+  - "left_hip_roll"
+  - "left_hip_pitch"
+  - "left_knee"
+  - "left_ankle"
+  - "right_hip_yaw"
+  - "right_hip_roll"
+  - "right_hip_pitch"
+  - "right_knee"
+  - "right_ankle"
+  - "torso"
+  - "left_shoulder_pitch"
+  - "left_shoulder_roll"
+  - "left_shoulder_yaw"
+  - "left_elbow"
+  - "right_shoulder_pitch"
+  - "right_shoulder_roll"
+  - "right_shoulder_yaw"
+  - "right_elbow"
+
+# Joint names in the target physics engine where policy is deployed (PhysX)
+target_joint_names:
+  - "left_hip_yaw"
+  - "right_hip_yaw"
+  - "torso"
+  - "left_hip_roll"
+  - "right_hip_roll"
+  - "left_shoulder_pitch"
+  - "right_shoulder_pitch"
+  - "left_hip_pitch"
+  - "right_hip_pitch"
+  - "left_shoulder_roll"
+  - "right_shoulder_roll"
+  - "left_knee"
+  - "right_knee"
+  - "left_shoulder_yaw"
+  - "right_shoulder_yaw"
+  - "left_ankle"
+  - "right_ankle"
+  - "left_elbow"
+  - "right_elbow"