Adds Factory contact-rich manipulation tasks to IsaacLab (#1520)

# Description

This MR adds new tasks for contact-rich manipulation based on the
Factory line of work. Tasks include peg insertion, gear meshing, and nut
threading.

## Type of change

- New feature (non-breaking change which adds functionality)
- This change requires a documentation update

## Screenshots

<img
src="https://github.com/user-attachments/assets/c8c32592-8972-4b1e-be7c-51b0313393bf"
alt="peg_insertion" height="200"/>
<img
src="https://github.com/user-attachments/assets/437baf31-f9cb-4a27-94ac-5f21e2309089"
alt="gear_meshing" height="200"/>
<img
src="https://github.com/user-attachments/assets/bccdf71b-cd5f-45f7-b4ca-71616885ef48"
alt="nut_threading" height="200"/>

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

---------
Signed-off-by: Kelly Guo <kellyg@nvidia.com>
Signed-off-by: Kelly Guo <kellyguo123@hotmail.com>
Co-authored-by: Iretiayo Akinola <iakinola@nvidia.com>
Co-authored-by: Kelly Guo <kellyg@nvidia.com>
Co-authored-by: Kelly Guo <kellyguo123@hotmail.com>

CONTRIBUTORS.md

@@ -50,6 +50,7 @@ Guidelines for modifications:
 * Haoran Zhou
 * HoJin Jeon
 * Hongwei Xiong
+* Iretiayo Akinola
 * Jan Kerner
 * Jean Tampon
 * Jia Lin Yuan
@@ -63,6 +64,7 @@ Guidelines for modifications:
 * Lorenz Wellhausen
 * Masoud Moghani
 * Michael Gussert
+* Michael Noseworthy
 * Muhong Guo
 * Nuralem Abizov
 * Oyindamola Omotuyi
@@ -91,3 +93,4 @@ Guidelines for modifications:
 * Gavriel State
 * Hammad Mazhar
 * Marco Hutter
+* Yashraj Narang

docs/source/overview/environments.rst

@@ -152,6 +152,39 @@ for the lift-cube environment:
 .. |cube-shadow-lstm-link| replace:: `Isaac-Repose-Cube-Shadow-OpenAI-LSTM-Direct-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/shadow_hand/shadow_hand_env_cfg.py>`__
 .. |cube-shadow-vis-link| replace:: `Isaac-Repose-Cube-Shadow-Vision-Direct-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/shadow_hand/shadow_hand_vision_env.py>`__
 
+Contact-rich Manipulation
+~~~~~~~~~~~~
+
+Environments based on contact-rich manipulation tasks such as peg insertion, gear meshing and nut-bolt fastening.
+
+These tasks share the same task configurations and control options. You can switch between them by specifying the task name.
+For example:
+
+* |factory-peg-link|: Peg insertion with the Franka arm
+* |factory-gear-link|: Gear meshing with the Franka arm
+* |factory-nut-link|: Nut-Bolt fastening with the Franka arm
+
+.. table::
+    :widths: 33 37 30
+
+    +--------------------+-------------------------+-----------------------------------------------------------------------------+
+    | World              | Environment ID          | Description                                                                 |
+    +====================+=========================+=============================================================================+
+    | |factory-peg|      | |factory-peg-link|      | Insert peg into the socket with the Franka robot                            |
+    +--------------------+-------------------------+-----------------------------------------------------------------------------+
+    | |factory-gear|     | |factory-gear-link|     | Insert and mesh gear into the base with other gears, using the Franka robot |
+    +--------------------+-------------------------+-----------------------------------------------------------------------------+
+    | |factory-nut|      | |factory-nut-link|      | Thread the nut onto the first 2 threads of the bolt, using the Franka robot |
+    +--------------------+-------------------------+-----------------------------------------------------------------------------+
+
+.. |factory-peg| image:: ../_static/tasks/factory/peg_insert.jpg
+.. |factory-gear| image:: ../_static/tasks/factory/gear_mesh.jpg
+.. |factory-nut| image:: ../_static/tasks/factory/nut_thread.jpg
+
+.. |factory-peg-link| replace:: `Isaac-Factory-PegInsert-Direct-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/factory/factory_env_cfg.py>`__
+.. |factory-gear-link| replace:: `Isaac-Factory-GearMesh-Direct-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/factory/factory_env_cfg.py>`__
+.. |factory-nut-link| replace:: `Isaac-Factory-NutThread-Direct-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/factory/factory_env_cfg.py>`__
+
 Locomotion
 ~~~~~~~~~~
 
@@ -369,6 +402,18 @@ Comprehensive List of Environments
       -
       - Manager Based
       - **rl_games** (PPO), **rsl_rl** (PPO), **skrl** (PPO), **sb3** (PPO)
+    * - Isaac-Factory-GearMesh-Direct-v0
+      -
+      - Direct
+      - **rl_games** (PPO)
+    * - Isaac-Factory-NutThread-Direct-v0
+      -
+      - Direct
+      - **rl_games** (PPO)
+    * - Isaac-Factory-PegInsert-Direct-v0
+      -
+      - Direct
+      - **rl_games** (PPO)
     * - Isaac-Franka-Cabinet-Direct-v0
       -
       - Direct

source/extensions/omni.isaac.lab_tasks/config/extension.toml

 [package]
 
 # Note: Semantic Versioning is used: https://semver.org/
-version = "0.10.15"
+version = "0.10.16"
 
 # Description
 title = "Isaac Lab Environments"

source/extensions/omni.isaac.lab_tasks/docs/CHANGELOG.rst

 Changelog
 ---------
 
+0.10.16 (2024-12-16)
+~~~~~~~~~~~~~~~~~~~~
+
+Added
+^^^^^
+
+* Added ``Factory-Direct-v0`` environment as a direct RL env that
+  implements contact-rich manipulation tasks including peg insertion,
+  gear meshing, and nut threading.
+
+
 0.10.15 (2024-12-16)
 ~~~~~~~~~~~~~~~~~~~~
 
 Added
 ^^^^^
+
 * Added ``Isaac-Reach-Franka-OSC-v0`` and ``Isaac-Reach-Franka-OSC-Play-v0``
   variations of the manager based reach environment that uses
   :class:`omni.isaac.lab.envs.mdp.actions.OperationalSpaceControllerAction`.
@@ -20,6 +32,7 @@ Added
 * Added ``Isaac-Stack-Cube-Franka-IK-Rel-v0`` and ``Isaac-Stack-Cube-Instance-Randomize-Franka-IK-Rel-v0`` environments
   as manager-based RL envs that implement a three cube stacking task.
 
+
 0.10.13 (2024-10-30)
 ~~~~~~~~~~~~~~~~~~~~
 
@@ -49,6 +62,7 @@ Added
 
 * Added feature extracted observation cartpole examples.
 
+
 0.10.10 (2024-10-25)
 ~~~~~~~~~~~~~~~~~~~~
 

source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/factory/__init__.py

+# Copyright (c) 2022-2024, The Isaac Lab Project Developers.
+# All rights reserved.
+#
+# SPDX-License-Identifier: BSD-3-Clause
+
+import gymnasium as gym
+
+from . import agents
+from .factory_env import FactoryEnv
+from .factory_env_cfg import FactoryTaskGearMeshCfg, FactoryTaskNutThreadCfg, FactoryTaskPegInsertCfg
+
+##
+# Register Gym environments.
+##
+
+gym.register(
+    id="Isaac-Factory-PegInsert-Direct-v0",
+    entry_point="omni.isaac.lab_tasks.direct.factory:FactoryEnv",
+    disable_env_checker=True,
+    kwargs={
+        "env_cfg_entry_point": FactoryTaskPegInsertCfg,
+        "rl_games_cfg_entry_point": f"{agents.__name__}:rl_games_ppo_cfg.yaml",
+    },
+)
+
+gym.register(
+    id="Isaac-Factory-GearMesh-Direct-v0",
+    entry_point="omni.isaac.lab_tasks.direct.factory:FactoryEnv",
+    disable_env_checker=True,
+    kwargs={
+        "env_cfg_entry_point": FactoryTaskGearMeshCfg,
+        "rl_games_cfg_entry_point": f"{agents.__name__}:rl_games_ppo_cfg.yaml",
+    },
+)
+
+gym.register(
+    id="Isaac-Factory-NutThread-Direct-v0",
+    entry_point="omni.isaac.lab_tasks.direct.factory:FactoryEnv",
+    disable_env_checker=True,
+    kwargs={
+        "env_cfg_entry_point": FactoryTaskNutThreadCfg,
+        "rl_games_cfg_entry_point": f"{agents.__name__}:rl_games_ppo_cfg.yaml",
+    },
+)

source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/factory/agents/__init__.py

+# Copyright (c) 2022-2024, The Isaac Lab Project Developers.
+# All rights reserved.
+#
+# SPDX-License-Identifier: BSD-3-Clause

source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/factory/agents/rl_games_ppo_cfg.yaml

+params:
+  seed: 0
+  algo:
+    name: a2c_continuous
+
+  env:
+    clip_actions: 1.0
+
+  model:
+    name: continuous_a2c_logstd
+
+  network:
+    name: actor_critic
+    separate: False
+
+    space:
+      continuous:
+        mu_activation: None
+        sigma_activation: None
+        mu_init:
+          name: default
+        sigma_init:
+          name: const_initializer
+          val: 0
+        fixed_sigma: False
+    mlp:
+      units: [512, 128, 64]
+      activation: elu
+      d2rl: False
+
+      initializer:
+        name: default
+      regularizer:
+        name: None
+
+    rnn:
+      name: lstm
+      units: 1024
+      layers: 2
+      before_mlp: True
+      concat_input: True
+      layer_norm: True
+
+  load_checkpoint: False
+  load_path: ""
+
+  config:
+    name: Factory
+    device: cuda:0
+    full_experiment_name: test
+    env_name: rlgpu
+    multi_gpu: False
+    ppo: True
+    mixed_precision: True
+    normalize_input: True
+    normalize_value: True
+    value_bootstrap: True
+    num_actors: 128
+    reward_shaper:
+      scale_value: 1.0
+    normalize_advantage: True
+    gamma: 0.995
+    tau: 0.95
+    learning_rate: 1.0e-4
+    lr_schedule: adaptive
+    schedule_type: standard
+    kl_threshold: 0.008
+    score_to_win: 20000
+    max_epochs: 200
+    save_best_after: 10
+    save_frequency: 100
+    print_stats: True
+    grad_norm: 1.0
+    entropy_coef: 0.0  # 0.0001  # 0.0
+    truncate_grads: True
+    e_clip: 0.2
+    horizon_length: 128
+    minibatch_size: 512  # batch size = num_envs * horizon_length; minibatch_size = batch_size / num_minibatches
+    mini_epochs: 4
+    critic_coef: 2
+    clip_value: True
+    seq_length: 128
+    bounds_loss_coef: 0.0001
+
+    central_value_config:
+      minibatch_size: 512
+      mini_epochs: 4
+      learning_rate: 1e-4
+      lr_schedule: adaptive
+      kl_threshold: 0.008
+      clip_value: True
+      normalize_input: True
+      truncate_grads: True
+
+      network:
+        name: actor_critic
+        central_value: True
+
+        mlp:
+          units: [512, 128, 64]
+          activation: elu
+          d2rl: False
+
+          initializer:
+            name: default
+          regularizer:
+            name: None
+
+        rnn:
+          name: lstm
+          units: 1024
+          layers: 2
+          before_mlp: True
+          concat_input: True
+          layer_norm: True
+
+    player:
+      deterministic: False

source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/factory/factory_control.py

+# Copyright (c) 2022-2024, The Isaac Lab Project Developers.
+# All rights reserved.
+#
+# SPDX-License-Identifier: BSD-3-Clause
+
+"""Factory: control module.
+
+Imported by base, environment, and task classes. Not directly executed.
+"""
+
+import math
+import torch
+
+import omni.isaac.core.utils.torch as torch_utils
+
+from omni.isaac.lab.utils.math import axis_angle_from_quat
+
+
+def compute_dof_torque(
+    cfg,
+    dof_pos,
+    dof_vel,
+    fingertip_midpoint_pos,
+    fingertip_midpoint_quat,
+    fingertip_midpoint_linvel,
+    fingertip_midpoint_angvel,
+    jacobian,
+    arm_mass_matrix,
+    ctrl_target_fingertip_midpoint_pos,
+    ctrl_target_fingertip_midpoint_quat,
+    task_prop_gains,
+    task_deriv_gains,
+    device,
+):
+    """Compute Franka DOF torque to move fingertips towards target pose."""
+    # References:
+    # 1) https://ethz.ch/content/dam/ethz/special-interest/mavt/robotics-n-intelligent-systems/rsl-dam/documents/RobotDynamics2018/RD_HS2018script.pdf
+    # 2) Modern Robotics
+
+    num_envs = cfg.scene.num_envs
+    dof_torque = torch.zeros((num_envs, dof_pos.shape[1]), device=device)
+    task_wrench = torch.zeros((num_envs, 6), device=device)
+
+    pos_error, axis_angle_error = get_pose_error(
+        fingertip_midpoint_pos=fingertip_midpoint_pos,
+        fingertip_midpoint_quat=fingertip_midpoint_quat,
+        ctrl_target_fingertip_midpoint_pos=ctrl_target_fingertip_midpoint_pos,
+        ctrl_target_fingertip_midpoint_quat=ctrl_target_fingertip_midpoint_quat,
+        jacobian_type="geometric",
+        rot_error_type="axis_angle",
+    )
+    delta_fingertip_pose = torch.cat((pos_error, axis_angle_error), dim=1)
+
+    # Set tau = k_p * task_pos_error - k_d * task_vel_error (building towards eq. 3.96-3.98)
+    task_wrench_motion = _apply_task_space_gains(
+        delta_fingertip_pose=delta_fingertip_pose,
+        fingertip_midpoint_linvel=fingertip_midpoint_linvel,
+        fingertip_midpoint_angvel=fingertip_midpoint_angvel,
+        task_prop_gains=task_prop_gains,
+        task_deriv_gains=task_deriv_gains,
+    )
+    task_wrench += task_wrench_motion
+
+    # Set tau = J^T * tau, i.e., map tau into joint space as desired
+    jacobian_T = torch.transpose(jacobian, dim0=1, dim1=2)
+    dof_torque[:, 0:7] = (jacobian_T @ task_wrench.unsqueeze(-1)).squeeze(-1)
+
+    # adapted from https://gitlab-master.nvidia.com/carbon-gym/carbgym/-/blob/b4bbc66f4e31b1a1bee61dbaafc0766bbfbf0f58/python/examples/franka_cube_ik_osc.py#L70-78
+    # roboticsproceedings.org/rss07/p31.pdf
+
+    # useful tensors
+    arm_mass_matrix_inv = torch.inverse(arm_mass_matrix)
+    jacobian_T = torch.transpose(jacobian, dim0=1, dim1=2)
+    arm_mass_matrix_task = torch.inverse(
+        jacobian @ torch.inverse(arm_mass_matrix) @ jacobian_T
+    )  # ETH eq. 3.86; geometric Jacobian is assumed
+    j_eef_inv = arm_mass_matrix_task @ jacobian @ arm_mass_matrix_inv
+    default_dof_pos_tensor = torch.tensor(cfg.ctrl.default_dof_pos_tensor, device=device).repeat((num_envs, 1))
+    # nullspace computation
+    distance_to_default_dof_pos = default_dof_pos_tensor - dof_pos[:, :7]
+    distance_to_default_dof_pos = (distance_to_default_dof_pos + math.pi) % (
+        2 * math.pi
+    ) - math.pi  # normalize to [-pi, pi]
+    u_null = cfg.ctrl.kd_null * -dof_vel[:, :7] + cfg.ctrl.kp_null * distance_to_default_dof_pos
+    u_null = arm_mass_matrix @ u_null.unsqueeze(-1)
+    torque_null = (torch.eye(7, device=device).unsqueeze(0) - torch.transpose(jacobian, 1, 2) @ j_eef_inv) @ u_null
+    dof_torque[:, 0:7] += torque_null.squeeze(-1)
+
+    # TODO: Verify it's okay to no longer do gripper control here.
+    dof_torque = torch.clamp(dof_torque, min=-100.0, max=100.0)
+    return dof_torque, task_wrench
+
+
+def get_pose_error(
+    fingertip_midpoint_pos,
+    fingertip_midpoint_quat,
+    ctrl_target_fingertip_midpoint_pos,
+    ctrl_target_fingertip_midpoint_quat,
+    jacobian_type,
+    rot_error_type,
+):
+    """Compute task-space error between target Franka fingertip pose and current pose."""
+    # Reference: https://ethz.ch/content/dam/ethz/special-interest/mavt/robotics-n-intelligent-systems/rsl-dam/documents/RobotDynamics2018/RD_HS2018script.pdf
+
+    # Compute pos error
+    pos_error = ctrl_target_fingertip_midpoint_pos - fingertip_midpoint_pos
+
+    # Compute rot error
+    if jacobian_type == "geometric":  # See example 2.9.8; note use of J_g and transformation between rotation vectors
+        # Compute quat error (i.e., difference quat)
+        # Reference: https://personal.utdallas.edu/~sxb027100/dock/quat.html
+
+        # Check for shortest path using quaternion dot product.
+        quat_dot = (ctrl_target_fingertip_midpoint_quat * fingertip_midpoint_quat).sum(dim=1, keepdim=True)
+        ctrl_target_fingertip_midpoint_quat = torch.where(
+            quat_dot.expand(-1, 4) >= 0, ctrl_target_fingertip_midpoint_quat, -ctrl_target_fingertip_midpoint_quat
+        )
+
+        fingertip_midpoint_quat_norm = torch_utils.quat_mul(
+            fingertip_midpoint_quat, torch_utils.quat_conjugate(fingertip_midpoint_quat)
+        )[
+            :, 0
+        ]  # scalar component
+        fingertip_midpoint_quat_inv = torch_utils.quat_conjugate(
+            fingertip_midpoint_quat
+        ) / fingertip_midpoint_quat_norm.unsqueeze(-1)
+        quat_error = torch_utils.quat_mul(ctrl_target_fingertip_midpoint_quat, fingertip_midpoint_quat_inv)
+
+        # Convert to axis-angle error
+        axis_angle_error = axis_angle_from_quat(quat_error)
+
+    if rot_error_type == "quat":
+        return pos_error, quat_error
+    elif rot_error_type == "axis_angle":
+        return pos_error, axis_angle_error
+
+
+def _get_delta_dof_pos(delta_pose, ik_method, jacobian, device):
+    """Get delta Franka DOF position from delta pose using specified IK method."""
+    # References:
+    # 1) https://www.cs.cmu.edu/~15464-s13/lectures/lecture6/iksurvey.pdf
+    # 2) https://ethz.ch/content/dam/ethz/special-interest/mavt/robotics-n-intelligent-systems/rsl-dam/documents/RobotDynamics2018/RD_HS2018script.pdf (p. 47)
+
+    if ik_method == "pinv":  # Jacobian pseudoinverse
+        k_val = 1.0
+        jacobian_pinv = torch.linalg.pinv(jacobian)
+        delta_dof_pos = k_val * jacobian_pinv @ delta_pose.unsqueeze(-1)
+        delta_dof_pos = delta_dof_pos.squeeze(-1)
+
+    elif ik_method == "trans":  # Jacobian transpose
+        k_val = 1.0
+        jacobian_T = torch.transpose(jacobian, dim0=1, dim1=2)
+        delta_dof_pos = k_val * jacobian_T @ delta_pose.unsqueeze(-1)
+        delta_dof_pos = delta_dof_pos.squeeze(-1)
+
+    elif ik_method == "dls":  # damped least squares (Levenberg-Marquardt)
+        lambda_val = 0.1  # 0.1
+        jacobian_T = torch.transpose(jacobian, dim0=1, dim1=2)
+        lambda_matrix = (lambda_val**2) * torch.eye(n=jacobian.shape[1], device=device)
+        delta_dof_pos = jacobian_T @ torch.inverse(jacobian @ jacobian_T + lambda_matrix) @ delta_pose.unsqueeze(-1)
+        delta_dof_pos = delta_dof_pos.squeeze(-1)
+
+    elif ik_method == "svd":  # adaptive SVD
+        k_val = 1.0
+        U, S, Vh = torch.linalg.svd(jacobian)
+        S_inv = 1.0 / S
+        min_singular_value = 1.0e-5
+        S_inv = torch.where(S > min_singular_value, S_inv, torch.zeros_like(S_inv))
+        jacobian_pinv = (
+            torch.transpose(Vh, dim0=1, dim1=2)[:, :, :6] @ torch.diag_embed(S_inv) @ torch.transpose(U, dim0=1, dim1=2)
+        )
+        delta_dof_pos = k_val * jacobian_pinv @ delta_pose.unsqueeze(-1)
+        delta_dof_pos = delta_dof_pos.squeeze(-1)
+
+    return delta_dof_pos
+
+
+def _apply_task_space_gains(
+    delta_fingertip_pose, fingertip_midpoint_linvel, fingertip_midpoint_angvel, task_prop_gains, task_deriv_gains
+):
+    """Interpret PD gains as task-space gains. Apply to task-space error."""
+
+    task_wrench = torch.zeros_like(delta_fingertip_pose)
+
+    # Apply gains to lin error components
+    lin_error = delta_fingertip_pose[:, 0:3]
+    task_wrench[:, 0:3] = task_prop_gains[:, 0:3] * lin_error + task_deriv_gains[:, 0:3] * (
+        0.0 - fingertip_midpoint_linvel
+    )
+
+    # Apply gains to rot error components
+    rot_error = delta_fingertip_pose[:, 3:6]
+    task_wrench[:, 3:6] = task_prop_gains[:, 3:6] * rot_error + task_deriv_gains[:, 3:6] * (
+        0.0 - fingertip_midpoint_angvel
+    )
+    return task_wrench

source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/factory/factory_env_cfg.py

+# Copyright (c) 2022-2024, The Isaac Lab Project Developers.
+# All rights reserved.
+#
+# SPDX-License-Identifier: BSD-3-Clause
+
+import omni.isaac.lab.sim as sim_utils
+from omni.isaac.lab.actuators.actuator_cfg import ImplicitActuatorCfg
+from omni.isaac.lab.assets import ArticulationCfg
+from omni.isaac.lab.envs import DirectRLEnvCfg
+from omni.isaac.lab.scene import InteractiveSceneCfg
+from omni.isaac.lab.sim import PhysxCfg, SimulationCfg
+from omni.isaac.lab.sim.spawners.materials.physics_materials_cfg import RigidBodyMaterialCfg
+from omni.isaac.lab.utils import configclass
+
+from .factory_tasks_cfg import ASSET_DIR, FactoryTask, GearMesh, NutThread, PegInsert
+
+OBS_DIM_CFG = {
+    "fingertip_pos": 3,
+    "fingertip_pos_rel_fixed": 3,
+    "fingertip_quat": 4,
+    "ee_linvel": 3,
+    "ee_angvel": 3,
+}
+
+STATE_DIM_CFG = {
+    "fingertip_pos": 3,
+    "fingertip_pos_rel_fixed": 3,
+    "fingertip_quat": 4,
+    "ee_linvel": 3,
+    "ee_angvel": 3,
+    "joint_pos": 7,
+    "held_pos": 3,
+    "held_pos_rel_fixed": 3,
+    "held_quat": 4,
+    "fixed_pos": 3,
+    "fixed_quat": 4,
+    "task_prop_gains": 6,
+    "ema_factor": 1,
+    "pos_threshold": 3,
+    "rot_threshold": 3,
+}
+
+
+@configclass
+class ObsRandCfg:
+    fixed_asset_pos = [0.001, 0.001, 0.001]
+
+
+@configclass
+class CtrlCfg:
+    ema_factor = 0.2
+
+    pos_action_bounds = [0.05, 0.05, 0.05]
+    rot_action_bounds = [1.0, 1.0, 1.0]
+
+    pos_action_threshold = [0.02, 0.02, 0.02]
+    rot_action_threshold = [0.097, 0.097, 0.097]
+
+    reset_joints = [1.5178e-03, -1.9651e-01, -1.4364e-03, -1.9761, -2.7717e-04, 1.7796, 7.8556e-01]
+    reset_task_prop_gains = [300, 300, 300, 20, 20, 20]
+    reset_rot_deriv_scale = 10.0
+    default_task_prop_gains = [100, 100, 100, 30, 30, 30]
+
+    # Null space parameters.
+    default_dof_pos_tensor = [-1.3003, -0.4015, 1.1791, -2.1493, 0.4001, 1.9425, 0.4754]
+    kp_null = 10.0
+    kd_null = 6.3246
+
+
+@configclass
+class FactoryEnvCfg(DirectRLEnvCfg):
+    decimation = 8
+    action_space = 6
+    # num_*: will be overwritten to correspond to obs_order, state_order.
+    observation_space = 21
+    state_space = 72
+    obs_order: list = ["fingertip_pos_rel_fixed", "fingertip_quat", "ee_linvel", "ee_angvel"]
+    state_order: list = [
+        "fingertip_pos",
+        "fingertip_quat",
+        "ee_linvel",
+        "ee_angvel",
+        "joint_pos",
+        "held_pos",
+        "held_pos_rel_fixed",
+        "held_quat",
+        "fixed_pos",
+        "fixed_quat",
+    ]
+
+    task_name: str = "peg_insert"  # peg_insert, gear_mesh, nut_thread
+    task: FactoryTask = FactoryTask()
+    obs_rand: ObsRandCfg = ObsRandCfg()
+    ctrl: CtrlCfg = CtrlCfg()
+
+    episode_length_s = 10.0  # Probably need to override.
+    sim: SimulationCfg = SimulationCfg(
+        device="cuda:0",
+        dt=1 / 120,
+        gravity=(0.0, 0.0, -9.81),
+        physx=PhysxCfg(
+            solver_type=1,
+            max_position_iteration_count=192,  # Important to avoid interpenetration.
+            max_velocity_iteration_count=1,
+            bounce_threshold_velocity=0.2,
+            friction_offset_threshold=0.01,
+            friction_correlation_distance=0.00625,
+            gpu_max_rigid_contact_count=2**23,
+            gpu_max_rigid_patch_count=2**23,
+            gpu_max_num_partitions=1,  # Important for stable simulation.
+        ),
+        physics_material=RigidBodyMaterialCfg(
+            static_friction=1.0,
+            dynamic_friction=1.0,
+        ),
+    )
+
+    scene: InteractiveSceneCfg = InteractiveSceneCfg(num_envs=128, env_spacing=2.0)
+
+    robot = ArticulationCfg(
+        prim_path="/World/envs/env_.*/Robot",
+        spawn=sim_utils.UsdFileCfg(
+            usd_path=f"{ASSET_DIR}/franka_mimic.usd",
+            activate_contact_sensors=True,
+            rigid_props=sim_utils.RigidBodyPropertiesCfg(
+                disable_gravity=True,
+                max_depenetration_velocity=5.0,
+                linear_damping=0.0,
+                angular_damping=0.0,
+                max_linear_velocity=1000.0,
+                max_angular_velocity=3666.0,
+                enable_gyroscopic_forces=True,
+                solver_position_iteration_count=192,
+                solver_velocity_iteration_count=1,
+                max_contact_impulse=1e32,
+            ),
+            articulation_props=sim_utils.ArticulationRootPropertiesCfg(
+                enabled_self_collisions=False,
+                solver_position_iteration_count=192,
+                solver_velocity_iteration_count=1,
+            ),
+            collision_props=sim_utils.CollisionPropertiesCfg(contact_offset=0.005, rest_offset=0.0),
+        ),
+        init_state=ArticulationCfg.InitialStateCfg(
+            joint_pos={
+                "panda_joint1": 0.00871,
+                "panda_joint2": -0.10368,
+                "panda_joint3": -0.00794,
+                "panda_joint4": -1.49139,
+                "panda_joint5": -0.00083,
+                "panda_joint6": 1.38774,
+                "panda_joint7": 0.0,
+                "panda_finger_joint2": 0.04,
+            },
+            pos=(0.0, 0.0, 0.0),
+            rot=(1.0, 0.0, 0.0, 0.0),
+        ),
+        actuators={
+            "panda_arm1": ImplicitActuatorCfg(
+                joint_names_expr=["panda_joint[1-4]"],
+                stiffness=0.0,
+                damping=0.0,
+                friction=0.0,
+                armature=0.0,
+                effort_limit=87,
+                velocity_limit=124.6,
+            ),
+            "panda_arm2": ImplicitActuatorCfg(
+                joint_names_expr=["panda_joint[5-7]"],
+                stiffness=0.0,
+                damping=0.0,
+                friction=0.0,
+                armature=0.0,
+                effort_limit=12,
+                velocity_limit=149.5,
+            ),
+            "panda_hand": ImplicitActuatorCfg(
+                joint_names_expr=["panda_finger_joint[1-2]"],
+                effort_limit=40.0,
+                velocity_limit=0.04,
+                stiffness=7500.0,
+                damping=173.0,
+                friction=0.1,
+                armature=0.0,
+            ),
+        },
+    )
+
+
+@configclass
+class FactoryTaskPegInsertCfg(FactoryEnvCfg):
+    task_name = "peg_insert"
+    task = PegInsert()
+    episode_length_s = 10.0
+
+
+@configclass
+class FactoryTaskGearMeshCfg(FactoryEnvCfg):
+    task_name = "gear_mesh"
+    task = GearMesh()
+    episode_length_s = 20.0
+
+
+@configclass
+class FactoryTaskNutThreadCfg(FactoryEnvCfg):
+    task_name = "nut_thread"
+    task = NutThread()
+    episode_length_s = 30.0

source/standalone/workflows/rl_games/play.py

@@ -155,7 +155,7 @@ def main():
             # convert obs to agent format
             obs = agent.obs_to_torch(obs)
             # agent stepping
-            actions = agent.get_action(obs, is_deterministic=True)
+            actions = agent.get_action(obs, is_deterministic=agent.is_deterministic)
             # env stepping
             obs, _, dones, _ = env.step(actions)
 

tools/per_test_timeouts.py

@@ -10,7 +10,7 @@ Any tests not listed here will use the default timeout.
 PER_TEST_TIMEOUTS = {
     "test_articulation.py": 200,
     "test_deformable_object.py": 200,
-    "test_environments.py": 1200,  # This test runs through all the environments for 100 steps each
+    "test_environments.py": 1500,  # This test runs through all the environments for 100 steps each
     "test_environment_determinism.py": 200,  # This test runs through many the environments for 100 steps each
     "test_env_rendering_logic.py": 300,
     "test_camera.py": 500,