Skip to content

K
KincoActuatorIsaacLab
Unverified Commit bff476a6 authored by Mayank Mittal's avatar Mayank Mittal Committed by GitHub
Browse files
Options

Cleans up the how-to documentation (#325) 

# Description

This MR reviews the how-tos documentation and ensures they are all
somewhat consistent.

## Type of change

- This change requires a documentation update

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./orbit.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there
parent cf7a65f3
Show whitespace changes
Inline Side-by-side
Showing with 198 additions and 167 deletions
docs/source/how-to/draw_markers.rst
View file @ bff476a6
Creating Markers in Orbit
=========================
Creating Visualization Markers
==============================
In this tutorial, we will explore how to create different types of markers using a Python script.
The script demonstrates the creation of markers with various shapes and visual properties.
.. currentmodule:: omni.isaac.orbit
Please ensure you have gone through the previous tutorials, especially creating an empty scene for a foundational understanding.
Visualization markers are useful to debug the state of the environment. They can be used to visualize
the frames, commands, and other information in the simulation.
While Isaac Sim provides its own :mod:`omni.isaac.debug_draw` extension, it is limited to rendering only
points, lines and splines. For cases, where you need to render more complex shapes, you can use the
:class:`markers.VisualizationMarkers` class.
The Code
~~~~~~~~
This guide is accompanied by a sample script ``markers.py`` in the ``orbit/source/standalone/demos`` directory.
The tutorial corresponds to the ``markers.py`` script in the ``orbit/source/standalone/demos`` directory.
Let's take a look at the Python script:
.. dropdown:: Code for markers.py
:icon: code
.. literalinclude:: ../../../source/standalone/demos/markers.py
.. literalinclude:: ../../../source/standalone/demos/markers.py
:language: python
:emphasize-lines: 49-97, 113, 114-145
:linenos:
The Code Explained
~~~~~~~~~~~~~~~~~~
Creating and spawning markers
-----------------------------
The :meth:`spawn_markers` function creates different types of markers with specified configurations.
For example, we include frames, arrows, cubes, spheres, cylinders, cones, and meshes.
The function returns a :obj:`VisualizationMarkers` object.
Configuring the markers
-----------------------
The :class:`~markers.VisualizationMarkersCfg` class provides a simple interface to configure
different types of markers. It takes in the following parameters:
- :attr:`~markers.VisualizationMarkersCfg.prim_path`: The corresponding prim path for the marker class.
- :attr:`~markers.VisualizationMarkersCfg.markers`: A dictionary specifying the different marker prototypes
handled by the class. The key is the name of the marker prototype and the value is its spawn configuration.
.. note::
In case the marker prototype specifies a configuration with physics properties, these are removed.
This is because the markers are not meant to be simulated.
Here we show all the different types of markers that can be configured. These range from simple shapes like
cones and spheres to more complex geometries like a frame or arrows. The marker prototypes can also be
configured from USD files.
.. literalinclude:: ../../../source/standalone/demos/markers.py
:language: python
:lines: 37-84
:linenos:
:lineno-start: 37
:lines: 49-97
:dedent:
Main simulation logic
---------------------
Drawing the markers
-------------------
The ``main`` function sets up the simulation context, camera view, and spawns lights into the stage.
It then creates instances of the markers and places them in a grid pattern.
The markers are rotated around the z-axis during the simulation for visualization purposes.
To draw the markers, we call the :class:`~markers.VisualizationMarkers.visualize` method. This method takes in
as arguments the pose of the markers and the corresponding marker prototypes to draw.
.. literalinclude:: ../../../source/standalone/demos/markers.py
:language: python
:lines: 86-111
:linenos:
:lineno-start: 86
:lines: 142-148
:dedent:
Executing the Script
~~~~~~~~~~~~~~~~~~~~
--------------------
To run the script, execute the following command:
To run the accompanying script, execute the following command:
.. code-block:: bash
./orbit.sh -p source/standalone/demos/markers.py
The simulation should start, and you can observe the different types of markers arranged in a grid pattern.
To stop the simulation, close the window, press the ``STOP`` button in the UI, or use ``Ctrl+C`` in the terminal.
The markers will rotating around their respective axes. Additionally every few rotations, they will
roll forward on the grid.
This tutorial provides a foundation for working with markers in Orbit.
You can further customize markers by adjusting their configurations and exploring additional options
available in the Orbit API.
To stop the simulation, close the window, or use ``Ctrl+C`` in the terminal.
docs/source/how-to/save_camera_output.rst
View file @ bff476a6
......@@ -6,44 +6,79 @@ Saving rendered images and 3D re-projection
.. currentmodule:: omni.isaac.orbit
This how-to demonstrates an efficient saving of rendered images and the projection of depth images into 3D Space.
This guide accompanied with the ``run_usd_camera.py`` script in the ``orbit/source/standalone/tutorials/04_sensors``
directory.
It is accompanied with the ``run_usd_camera.py`` script in the ``orbit/source/standalone/tutorials/04_sensors``
directory. For an introduction to sensors, please check the :ref:`tutorial-add-sensors-on-robot` tutorials.
.. dropdown:: Code for run_usd_camera.py
:icon: code
.. literalinclude:: ../../../source/standalone/tutorials/04_sensors/run_usd_camera.py
:language: python
:emphasize-lines: 137-139, 172-196, 200-204, 214-232
:linenos:
Saving the Images
-----------------
Saving using Replicator Basic Writer
------------------------------------
To save the images, we use the basic write class from Omniverse Replicator. This class allows us to save the
To save camera outputs, we use the basic write class from Omniverse Replicator. This class allows us to save the
images in a numpy format. For more information on the basic writer, please check the
`documentation <https://docs.omniverse.nvidia.com/extensions/latest/ext_replicator/writer_examples.html>`_.
.. literalinclude:: ../../../source/standalone/tutorials/04_sensors/run_usd_camera.py
:language: python
:lines: 135-137
:linenos:
:lineno-start: 135
:lines: 137-139
:dedent:
While stepping the simulator, the images can be saved to the defined folder.
Since the BasicWriter only supports saving data using NumPy format, we first need to convert the PyTorch sensors
to NumPy arrays before packing them in a dictionary.
While stepping the simulator, the images can be saved to the defined folder. Since the BasicWriter only supports
saving data using NumPy format, we first need to convert the PyTorch sensors to NumPy arrays before packing
them in a dictionary.
.. literalinclude:: ../../../source/standalone/tutorials/04_sensors/run_usd_camera.py
:language: python
:lines: 172-193
:linenos:
:lineno-start: 172
:lines: 172-192
:dedent:
After this step, we can save the images using the BasicWriter.
.. literalinclude:: ../../../source/standalone/tutorials/04_sensors/run_usd_camera.py
:language: python
:lines: 193-196
:dedent:
Projection into 3D Space
------------------------
In addition, we provide utilities to project the depth image into 3D Space.
The re-projection operations are done using torch which allows us to use the GPU for faster computation.
The resulting point cloud is visualized using the :mod:`omni.isaac.debug_draw` extension from Isaac Sim.
We include utilities to project the depth image into 3D Space. The re-projection operations are done using
PyTorch operations which allows faster computation.
.. literalinclude:: ../../../source/standalone/tutorials/04_sensors/run_usd_camera.py
:language: python
:lines: 197-229
:linenos:
:lineno-start: 197
:lines: 200-204
:dedent:
The resulting point cloud can be visualized using the :mod:`omni.isaac.debug_draw` extension from Isaac Sim.
This makes it easy to visualize the point cloud in the 3D space.
.. literalinclude:: ../../../source/standalone/tutorials/04_sensors/run_usd_camera.py
:language: python
:lines: 214-232
:dedent:
Executing the script
--------------------
To run the accompanying script, execute the following command:
.. code-block:: bash
./orbit.sh -p source/standalone/tutorials/04_sensors/run_usd_camera.py --save --draw
The simulation should start, and you can observe different objects falling down. An output folder will be created
in the ``orbit/source/standalone/tutorials/04_sensors`` directory, where the images will be saved. Additionally,
you should see the point cloud in the 3D space drawn on the viewport.
To stop the simulation, close the window, press the ``STOP`` button in the UI, or use ``Ctrl+C`` in the terminal.
docs/source/how-to/wrap_rl_env.rst
View file @ bff476a6
.. _how-to-env-wrappers:
Using environment wrappers
==========================
Wrapping environments
=====================
.. currentmodule:: omni.isaac.orbit
Environment wrappers are a way to modify the behavior of an environment without modifying the environment itself.
This can be used to apply functions to modify observations or rewards, record videos, enforce time limits, etc.
A detailed description of the API is available in the :class:`gymnasium.Wrapper` class.
At present, all RL environments inheriting from the :class:`omni.isaac.orbit.envs.RLTaskEnv` class
At present, all RL environments inheriting from the :class:`~envs.RLTaskEnv` class
are compatible with :class:`gymnasium.Wrapper`, since the base class implements the :class:`gymnasium.Env` interface.
In order to wrap an environment, you need to first initialize the base environment. After that, you can
wrap it with as many wrappers as you want by calling ``env = wrapper(env, *args, **kwargs)`` repeatedly.
......@@ -68,7 +71,7 @@ To use the wrapper, you need to first install ``ffmpeg``. On Ubuntu, you can ins
The viewport camera used for rendering is the default camera in the scene called ``"/OmniverseKit_Persp"``.
The camera's pose and image resolution can be configured through the
:class:`omni.isaac.orbit.envs.ViewerCfg` class.
:class:`~envs.ViewerCfg` class.
.. dropdown:: Default parameters of the ViewerCfg class:
......@@ -125,7 +128,7 @@ Every learning framework has its own API for interacting with environments. For
`Stable-Baselines3`_ library uses the `gym.Env <https://gymnasium.farama.org/api/env/>`_
interface to interact with environments. However, libraries like `RL-Games`_ or `RSL-RL`_
use their own API for interfacing with a learning environments. Since there is no one-size-fits-all
solution, we do not base the :class:`RLTaskEnv` class on any particular learning framework's
solution, we do not base the :class:`~envs.RLTaskEnv` class on any particular learning framework's
environment definition. Instead, we implement wrappers to make it compatible with the learning
framework's environment definition.
......
docs/source/how-to/write_articulation_cfg.rst
View file @ bff476a6
.. _how-to-create-articulation-config:
.. _how-to-write-articulation-config:
Creating an Articulation
========================
In this tutorial, we move beyond the use of pre-built Articulations such as
Anymal and Franka, focusing instead on the steps required to integrate
custom robots into Orbit. The tutorial provides a
step-by-step guide on importing a robot design in either USD format and
spawning it in Orbit as an :class:`Articulation`.
Writing an Asset Configuration
==============================
.. TODO: Talk about how to import via URDF
.. currentmodule:: omni.isaac.orbit
This guide walks through the process of creating an :class:`~assets.ArticulationCfg`.
The :class:`~assets.ArticulationCfg` is a configuration object that defines the
properties of an :class:`~assets.Articulation` in Orbit.
What is a Cartpole?
~~~~~~~~~~~~~~~~~~~
Cartpole, a variation of the inverted pendulum problem
(https://en.wikipedia.org/wiki/Inverted_pendulum), serves as a practical
example for learning traditional control and RL. The cartpole has a single
controllable degree of freedom (DOF) at the joint between the cart and
the rail. The attached pole has 1 DOF that allows it to rotate freely.
.. TODO: Add isaac sim screenshot and replace GIF with a webdb
In :ref:`tutorial-create-base-env` participants will learn to control the
pole to stabilize the cart, but this tutorial focuses on merely constructing
the :class:`ArticulationCfg` that defines the cartpole.
.. note::
The Code
~~~~~~~~
While we only cover the creation of an :class:`~assets.ArticulationCfg` in this guide,
the process is similar for creating any other asset configuration object.
In Orbit, we define an :class:`Articulation` by constructing its
configuration :class:`ArticulationCfg`. In the following sections we will break
down each part of the configuration.
We will use the Cartpole example to demonstrate how to create an :class:`~assets.ArticulationCfg`.
The Cartpole is a simple robot that consists of a cart with a pole attached to it. The cart
is free to move along a rail, and the pole is free to rotate about the cart.
.. dropdown:: Code for USD import configuration
.. dropdown:: Code for Cartpole configuration
:icon: code
.. literalinclude:: ../../../source/extensions/omni.isaac.orbit/omni/isaac/orbit/assets/config/cartpole.py
......@@ -42,88 +27,90 @@ down each part of the configuration.
:linenos:
The Code Explained
~~~~~~~~~~~~~~~~~~
Defining the spawn configuration
--------------------------------
Importing Cartpole's USD
^^^^^^^^^^^^^^^^^^^^^^^^
As explained in :ref:`tutorial-spawn-prims` tutorials, the spawn configuration defines
the properties of the assets to be spawned. This spawning may happen procedurally, or
through an existing asset file (e.g. USD or URDF). In this example, we will spawn the
Cartpole from a USD file.
The next chunk of code handles the USD import of the Cartpole:
When spawning an asset from a USD file, we define its :class:`~sim.spawners.from_files.UsdFileCfg`.
This configuration object takes in the following parameters:
* Defining the USD file path from which to spawn the Cartpole
* Defining the rigid body properties of the Cartpole
* Defining properties of the root of the Cartpole
* :class:`~sim.spawners.from_files.UsdFileCfg.usd_path`: The USD file path to spawn from
* :class:`~sim.spawners.from_files.UsdFileCfg.rigid_props`: The properties of the articulation's root
* :class:`~sim.spawners.from_files.UsdFileCfg.articulation_props`: The properties of all the articulation's links
.. dropdown:: Code for USD import configuration
:icon: code
The last two parameters are optional. If not specified, they are kept at their default values in the USD file.
.. literalinclude:: ../../../source/extensions/omni.isaac.orbit/omni/isaac/orbit/assets/config/cartpole.py
.. literalinclude:: ../../../source/extensions/omni.isaac.orbit/omni/isaac/orbit/assets/config/cartpole.py
:language: python
:start-after: # USD file configuration
:end-before: # Initial state definition
.. note::
To import articulation from a URDF file instead of a USD file, use ``UrdfFileCfg`` found in
``source/extensions/omni.isaac.orbit/omni/isaac/orbit/sim/spawners/from_files/from_file_cfg``
and replace ``usd_path`` argument with ``urdf_path``. For more details, see the API documentation.
:lines: 17-33
:dedent:
.. TODO: Either add an example of this here or make a separate tutorial
To import articulation from a URDF file instead of a USD file, you can replace the
:class:`~sim.spawners.from_files.UsdFileCfg` with a :class:`~sim.spawners.from_files.UrdfFileCfg`.
For more details, please check the API documentation.
Defining Cartpole's USD File Path
"""""""""""""""""""""""""""""""""
First we define the path the Cartpole USD file will be loaded from. In this
case ``cartpole.usd`` is included in the Nucleus server.
Defining the initial state
--------------------------
.. TODO: Document Nucleus server somewhere or link it if docs exist
Every asset requires defining their initial or *default* state in the simulation through its configuration.
This configuration is stored into the asset's default state buffers that can be accessed when the asset's
state needs to be reset.
.. literalinclude:: ../../../source/extensions/omni.isaac.orbit/omni/isaac/orbit/assets/config/cartpole.py
:language: python
:start-after: # Location of USD file
:end-before: # Rigid body properties
Defining Cartpole's Rigid Body Properties
"""""""""""""""""""""""""""""""""""""""""
The rigid body properties define how the cartpole will interact with its
environment. The settings we want to modify in this example are:
.. note::
The initial state of an asset is defined w.r.t. its local environment frame. This then needs to
be transformed into the global simulation frame when resetting the asset's state. For more
details, please check the :ref:`tutorial-interact-articulation` tutorial.
* The rigid body to be enabled
* | the maximum values for linear and angular velocity and depenetration
| velocity which defines the speed at which objects in collision with one
| another move away from one another
* The Gyroscopic forces on our cartpole to be enabled
.. TODO: Either go into more detail here, or add tutorial on rigid body properties
For an articulation, the :class:`~assets.ArticulationCfg.InitialStateCfg` object defines the
initial state of the root of the articulation and the initial state of all its joints. In this
example, we will spawn the Cartpole at the origin of the XY plane at a Z height of 2.0 meters.
Meanwhile, the joint positions and velocities are set to 0.0.
.. literalinclude:: ../../../source/extensions/omni.isaac.orbit/omni/isaac/orbit/assets/config/cartpole.py
:language: python
:start-after: # Rigid body properties
:end-before: # Articulation root properties
:lines: 34-36
:dedent:
Defining the actuator configuration
-----------------------------------
Defining the Initial State of Cartpole
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Actuators are a crucial component of an articulation. Through this configuration, it is possible
to define the type of actuator model to use. We can use the internal actuator model provided by
the physics engine (i.e. the implicit actuator model), or use a custom actuator model which is
governed by a user-defined system of equations (i.e. the explicit actuator model).
For more details on actuators, see :ref:`feature-actuators`.
The :class:`InitialStateCfg` object defines the initial state of the root of
an articulation in addition to the initial state of any joints. In this
example, we will spawn the Cartpole at the origin of the XY plane at a Z height
of 2.0 meters. The cart's joints will default to 0.0 as defined by the ``joint_pos``
parameter.
The cartpole's articulation has two actuators, one corresponding to its each joint:
``cart_to_pole`` and ``slider_to_cart``. We use two different actuator models for these actuators as
an example. However, since they are both using the same actuator model, it is possible
to combine them into a single actuator model.
.. literalinclude:: ../../../source/extensions/omni.isaac.orbit/omni/isaac/orbit/assets/config/cartpole.py
.. dropdown:: Actuator model configuration with separate actuator models
:icon: code
.. literalinclude:: ../../../source/extensions/omni.isaac.orbit/omni/isaac/orbit/assets/config/cartpole.py
:language: python
:start-after: # Initial state definition
:end-before: # Actuators definition
:lines: 37-47
:dedent:
Defining the Cartpole's Actuators
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The cartpole articulation has two actuators, one corresponding to each joint
``cart_to_pole`` and ``slider_to_cart``. for more details on actuators, see
:ref:`feature-actuators`.
.. dropdown:: Actuator model configuration with a single actuator model
:icon: code
.. literalinclude:: ../../../source/extensions/omni.isaac.orbit/omni/isaac/orbit/assets/config/cartpole.py
:language: python
:start-after: # Actuators definition
:end-before: # End cartpole articulation configuration
.. code-block:: python
actuators={
"all_joints": ImplicitActuatorCfg(
joint_names_expr=[".*"],
effort_limit=400.0,
velocity_limit=100.0,
stiffness={"slider_to_cart": 0.0, "cart_to_pole": 0.0},
damping={"slider_to_cart": 10.0, "cart_to_pole": 0.0},
),
},
docs/source/tutorials/01_assets/run_articulation.rst
View file @ bff476a6
......@@ -43,7 +43,7 @@ properties.
For the cart-pole, we use its pre-defined configuration object, which is an instance of the
:class:`assets.ArticulationCfg` class. This class contains information about the articulation's spawning strategy,
default initial state, actuator models for different joints, and other meta-information. A deeper-dive into how to
create this configuration object is provided in the :ref:`how-to-create-articulation-config` tutorial.
create this configuration object is provided in the :ref:`how-to-write-articulation-config` tutorial.
As seen in the previous tutorial, we can spawn the articulation into the scene in a similar fashion by creating
an instance of the :class:`assets.Articulation` class by passing the configuration object to its constructor.
......
source/extensions/omni.isaac.orbit/omni/isaac/orbit/assets/config/cartpole.py
View file @ bff476a6
......@@ -13,13 +13,9 @@ from omni.isaac.orbit.actuators import ImplicitActuatorCfg
from omni.isaac.orbit.assets import ArticulationCfg
from omni.isaac.orbit.utils.assets import ISAAC_ORBIT_NUCLEUS_DIR
# Cartpole articulation configuration
CARTPOLE_CFG = ArticulationCfg(
# USD file configuration
spawn=sim_utils.UsdFileCfg(
# Location of USD file
usd_path=f"{ISAAC_ORBIT_NUCLEUS_DIR}/Robots/Classic/Cartpole/cartpole.usd",
# Rigid body properties
rigid_props=sim_utils.RigidBodyPropertiesCfg(
rigid_body_enabled=True,
max_linear_velocity=1000.0,
......@@ -27,7 +23,6 @@ CARTPOLE_CFG = ArticulationCfg(
max_depenetration_velocity=100.0,
enable_gyroscopic_forces=True,
),
# Articulation root properties
articulation_props=sim_utils.ArticulationRootPropertiesCfg(
enabled_self_collisions=False,
solver_position_iteration_count=4,
......@@ -36,11 +31,9 @@ CARTPOLE_CFG = ArticulationCfg(
stabilization_threshold=0.001,
),
),
# Initial state definition
init_state=ArticulationCfg.InitialStateCfg(
pos=(0.0, 0.0, 2.0), joint_pos={"slider_to_cart": 0.0, "cart_to_pole": 0.0}
),
# Actuators definition
actuators={
"cart_actuator": ImplicitActuatorCfg(
joint_names_expr=["slider_to_cart"],
......@@ -53,5 +46,4 @@ CARTPOLE_CFG = ArticulationCfg(
joint_names_expr=["cart_to_pole"], effort_limit=400.0, velocity_limit=100.0, stiffness=0.0, damping=0.0
),
},
# End cartpole articulation configuration
)
source/standalone/demos/markers.py
View file @ bff476a6
......@@ -46,8 +46,8 @@ from omni.isaac.orbit.utils.assets import ISAAC_NUCLEUS_DIR, ISAAC_ORBIT_NUCLEUS
from omni.isaac.orbit.utils.math import quat_from_angle_axis
def spawn_markers():
"""Spawns markers with various different shapes."""
def define_markers() -> VisualizationMarkers:
"""Define markers with various different shapes."""
marker_cfg = VisualizationMarkersCfg(
prim_path="/Visuals/myMarkers",
markers={
......@@ -110,7 +110,7 @@ def main():
cfg.func("/World/Light", cfg)
# create markers
my_visualizer = spawn_markers()
my_visualizer = define_markers()
# define a grid of positions where the markers should be placed
num_markers_per_type = 5
......@@ -143,6 +143,9 @@ def main():
marker_orientations = quat_from_angle_axis(yaw, torch.tensor([0.0, 0.0, 1.0]))
# visualize
my_visualizer.visualize(marker_locations, marker_orientations, marker_indices=marker_indices)
# roll corresponding indices to show how marker prototype can be changed
if yaw[0].item() % (0.5 * torch.pi) < 0.01:
marker_indices = torch.roll(marker_indices, 1)
# perform step
sim.step()
# increment yaw
......
source/standalone/tutorials/04_sensors/run_usd_camera.py
View file @ bff476a6
......@@ -75,7 +75,7 @@ def define_sensor() -> Camera:
update_period=0,
height=480,
width=640,
data_types=["rgb", "distance_to_image_plane", "normals", "motion_vectors"],
data_types=["rgb", "distance_to_image_plane", "normals"],
spawn=sim_utils.PinholeCameraCfg(
focal_length=24.0, focus_distance=400.0, horizontal_aperture=20.955, clipping_range=(0.1, 1.0e5)
),
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment