Unverified Commit eb75c536 authored by Mayank Mittal's avatar Mayank Mittal Committed by GitHub

Adapts tutorial for interactive scene (#282)

# Description

This MR goes over the scene tutorial and makes it consistent with the
articulation tutorial.

## Type of change

- This change requires a documentation update

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./orbit.sh --format`
- [x] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there
parent 79fec2ff
.. _interactive-scene:
.. _tutorial-interactive-scene:
Using InteractiveScene
======================
In previous tutorials, we've used :meth:`spawn` to manually spawn assets,
but in this tutorial we introduce :class:`InteractiveScene` and its
associated configuration class :class:`InteractiveSceneCfg`.
Using the Interactive Scene
===========================
:class:`InteractiveScene` provides a few benefits over using assets'
:meth:`spawn` methods directly:
.. currentmodule:: omni.isaac.orbit
* collects all of the assets in a single configuration object which makes them easier
to manage
* enables user-friendly cloning of scene elements for multiple environments
* user doesn't need to call :meth:`spawn` for each asset - this is handled implicitly
So far in the tutorials, we manually spawned assets into the simulation and created
object instances to interact with them. However, as the complexity of the scene
increases, it becomes tedious to perform these tasks manually. In this tutorial,
we will introduce the :class:`scene.InteractiveScene` class, which provides a convenient
interface for spawning prims and managing them in the simulation.
At a high-level, the interactive scene is a collection of scene entities. Each entity
can be either a non-interactive prim (e.g. ground plane, light source), an interactive
prim (e.g. articulation, rigid object), or a sensor (e.g. camera, lidar). The interactive
scene provides a convenient interface for spawning these entities and managing them
in the simulation.
Compared the manual approach, it provides the following benefits:
* Alleviates the user needing to spawn each asset separately as this is handled implicitly.
* Enables user-friendly cloning of scene prims for multiple environments.
* Collects all the scene entities into a single object, which makes them easier to manage.
In this tutorial, we take the cartpole example from the :ref:`tutorial-interact-articulation`
tutorial and replace the ``design_scene`` function with an :class:`scene.InteractiveScene` object.
While it may seem like overkill to use the interactive scene for this simple example, it will
become more useful in the future as more assets and sensors are added to the scene.
We will implement an :class:`InteractiveSceneCfg` to design a simple scene
for the Cartpole, which consists of a ground
plane, light, and the cartpole :class:`Articulation`.
The Code
~~~~~~~~
This tutorial corresponds to the ``scene_creation.py`` script within
``orbit/source/standalone/tutorials``.
``orbit/source/standalone/tutorials/03_scene``.
.. dropdown:: :fa:`eye,mr-1` Code for ``scene_creation.py``
.. dropdown:: Code for scene_creation.py
:icon: code
.. literalinclude:: ../../../../source/standalone/tutorials/03_scene/scene_creation.py
:language: python
:emphasize-lines: 51-64, 69-71, 92-93, 100-101, 106-107, 117-119
:linenos:
The Code Explained
~~~~~~~~~~~~~~~~~~
We compose our scene configuration by sub-classing :class:`InteractiveSceneCfg`. We
then add the elements we want as attributes of the class, which is a common pattern
for configuration classes in Orbit. In this case, we add a
ground plane, light, and cartpole. The names of the attributes (``ground``,
``robot``, ``dome_light``, ``distant_light``) are used as keys to access the
corresponding assets in the :class:`InteractiveScene` object.
.. literalinclude:: ../../../../source/extensions/omni.isaac.orbit_tasks/omni/isaac/orbit_tasks/classic/cartpole/cartpole_scene.py
:language: python
:linenos:
Within the config elements of :class:`CartpoleSceneCfg`, notice that we pass
in a few arguments. We will describe these briefly as they are fundamental to
how the scene interface works - refer to in-code documentation for more depth.
While the code is similar to the previous tutorial, there are a few key differences
that we will go over in detail.
* **prim_path**: The USD layer to associate the asset with
* **spawn**: The configuration object for spawning
* **init_state**: The desired initial pose of the asset. Defaults to identity if unspecified
Scene configuration
-------------------
You can find more about additional arguments by looking directly at the docstrings for
:class:`AssetBaseCfg`, :class:`RigidObjectCfg` and :class:`ArticulationCfg`.
The scene is composed of a collection of entities, each with their own configuration.
These are specified in a configuration class that inherits from :class:`scene.InteractiveSceneCfg`.
The configuration class is then passed to the :class:`scene.InteractiveScene` constructor
to create the scene.
:class:`InteractiveSceneCfg` is simply a collection of
asset configurations, so add new elements as you see fit
(including sensors, markers, terrain, etc.) you've learned to utilize in previous tutorials.
For the cartpole example, we specify the same scene as in the previous tutorial, but list
them now in the configuration class :class:`CartpoleSceneCfg` instead of manually spawning them.
Setup Simulator and Spawn Scene
-------------------------------
.. literalinclude:: ../../../../source/standalone/tutorials/03_scene/scene_creation.py
:language: python
:lines: 51-64
:linenos:
:lineno-start: 51
The variable names in the configuration class are used as keys to access the corresponding
entity from the :class:`scene.InteractiveScene` object. For example, the cartpole can
be accessed via ``scene["cartpole"]``. However, we will get to that later. First, let's
look at how individual scene entities are configured.
Similar to how a rigid object and articulation were configured in the previous tutorials,
the configurations are specified using a configuration class. However, there is a key
difference between the configurations for the ground plane and light source and the
configuration for the cartpole. The ground plane and light source are non-interactive
prims, while the cartpole is an interactive prim. This distinction is reflected in the
configuration classes used to specify them. The configurations for the ground plane and
light source are specified using an instance of the :class:`assets.AssetBaseCfg` class
while the cartpole is configured using an instance of the :class:`assets.ArticulationCfg`.
Anything that is not an interactive prim (i.e., neither an asset nor a sensor) is not
*handled* by the scene during simulation steps.
Another key difference to note is in the specification of the prim paths for the
different prims:
* Ground plane: ``/World/defaultGroundPlane``
* Light source: ``/World/Light``
* Cartpole: ``{ENV_REGEX_NS}/Robot``
As we learned earlier, Omniverse creates a graph of prims in the USD stage. The prim
paths are used to specify the location of the prim in the graph. The ground plane and
light source are specified using absolute paths, while the cartpole is specified using
a relative path. The relative path is specified using the ``ENV_REGEX_NS`` variable,
which is a special variable that is replaced with the environment name during scene creation.
Any entity that has the ``ENV_REGEX_NS`` variable in its prim path will be cloned for each
environment. This path is replaced by the scene object with ``/World/envs/env_{i}`` where
``i`` is the environment index.
Scene instantiation
-------------------
Unlike before where we called the ``design_scene`` function to create the scene, we now
create an instance of the :class:`scene.InteractiveScene` class and pass in the configuration
object to its constructor. While creating the configuration instance of ``CartpoleSceneCfg``
we specify how many environment copies we want to create using the ``num_envs`` argument.
This will be used to clone the scene for each environment.
.. literalinclude:: ../../../../source/standalone/tutorials/03_scene/scene_creation.py
:language: python
:start-after: # Main
:end-before: # Extract cartpole from InteractiveScene
:lines: 117-119
:linenos:
:lineno-start: 117
Accessing Scene Elements
Accessing scene elements
------------------------
Individual scene elements can then be accessed from the :class:`InteractiveScene` via
the different asset groups: ``articulations``, ``rigid_objects``, ``sensors``,
``extras`` (where lights are found for instance). Each of these is a dictionary with the keys being assigned based
on the object instance name.
In the example script we access our cartpole :class:`Articulation` here using the ``"robot"`` key:
Similar to how entities were accessed from a dictionary in the previous tutorials, the
scene elements can be accessed from the :class:`InteractiveScene` object using the
``[]`` operator. The operator takes in a string key and returns the corresponding
entity. The key is specified through the configuration class for each entity. For example,
the cartpole is specified using the key ``"cartpole"`` in the configuration class.
.. literalinclude:: ../../../../source/standalone/tutorials/03_scene/scene_creation.py
:language: python
:start-after: # Extract cartpole from InteractiveScene
:end-before: # Simulation loop
:lines: 69-71
:linenos:
:lineno-start: 69
Simulation Loop
---------------
Running the simulation loop
---------------------------
.. literalinclude:: ../../../../source/standalone/tutorials/03_scene/scene_creation.py
:language: python
......@@ -91,46 +139,34 @@ Simulation Loop
:start-after: # Simulation loop
:end-before: # End simulation loop
The rest of the script should look familiar to previous scripts that interfaced with :class:`Articulation`,
with a few small differences:
* :meth:`Articulation.set_joint_position_target` and :meth:`Articulation.set_joint_velocity_target`
in combination with :meth:`InteractiveScene.write_data_to_sim`
are used instead of :meth:`Articulation.write_joint_data_to_sim` to set the desired position and velocity targets
without writing them to the simulation.
* :meth:`InteractiveScene.update` is used in place of :meth:`Articulation.update`
The rest of the script looks similar to previous scripts that interfaced with :class:`assets.Articulation`,
with a few small differences in the methods called:
Under the hood, ``InteractiveScene`` calls the ``update`` and ``write_data_to_sim`` for each asset in the scene,
so you only need to call these once per simulation step.
* :meth:`assets.Articulation.reset` ⟶ :meth:`scene.InteractiveScene.reset`
* :meth:`assets.Articulation.write_data_to_sim` ⟶ :meth:`scene.InteractiveScene.write_data_to_sim`
* :meth:`assets.Articulation.update` ⟶ :meth:`scene.InteractiveScene.update`
Cloning
-------
Under the hood, the methods of :class:`scene.InteractiveScene` call the corresponding
methods of the entities in the scene.
As mentioned previously, one of the key benefits of using :class:`InteractiveScene`
is its ability to handle cloning of assets seamlessly with the only user input
being the ``num_envs`` (The number of desired environments to spawn). The spacing between
environments is also configurable via ``env_spacing``.
We will exercise this below by passing in ``--num_envs 32`` to the tutorial script to spawn 32 cartpoles.
The Code Execution
~~~~~~~~~~~~~~~~~~
Now that we have gone through the code, let's run the script and see the result:
Let's run the script to simulate 32 cartpoles in the scene. We can do this by passing
the ``--num_envs`` argument to the script.
.. code-block:: bash
./orbit.sh -p source/standalone/tutorials/03_scene/scene_creation.py --num_envs 32
This should open a stage with 32 cartpoles swinging around randomly. You can use the
mouse to rotate the camera and the arrow keys to move around the scene.
This should open a stage with 32 cartpoles. The simulation should be
playing with the poles of each cartpole balancing vertically.
In this tutorial we saw how to use :class:`InteractiveScene` to create a
In this tutorial, we saw how to use :class:`scene.InteractiveScene` to create a
scene with multiple assets. We also saw how to use the ``num_envs`` argument
to clone the scene for multiple environments.
.. note::
There are many more examples of other ``InteractiveSceneCfg`` in the tasks found in
``source/extensions/omni.isaac.orbit_tasks/omni/isaac/orbit_tasks`` for
reference.
There are many more example usages of the :class:`scene.InteractiveSceneCfg` in the tasks found
under the ``omni.isaac.orbit_tasks`` extension. Please check out the source code to see
how they are used for more complex scenes.
......@@ -76,7 +76,7 @@ implementing a :class:`InteractiveSceneCfg`. This will then be used to construct
a :class:`InteractiveScene` which handles spawning of the objects in the scene.
In this tutorial, we will be using the configuration from ``cartpole_scene.py``.
See :ref:`interactive-scene` for a tutorial on how to create it.
See :ref:`tutorial-interactive-scene` for a tutorial on how to create it.
The scene used here consists of a ground plane, the cartpole and some lights.
......
......@@ -78,7 +78,7 @@ The first step in creating a new environment is to design the scene in which the
The scene used in this tutorial is the same one used in the tutorial :ref:`creating-base-env`, so we won't
go over it in detail again here.
Also see :ref:`interactive-scene` for even more details on scene creation.
Also see :ref:`tutorial-interactive-scene` for even more details on scene creation.
Designing the Action and Observation Spaces
-------------------------------------------
......
......@@ -11,7 +11,6 @@ import gymnasium as gym
from . import agents
from .cartpole_env_cfg import CartpoleEnvCfg
from .cartpole_scene import CartpoleSceneCfg
##
# Register Gym environments.
......
......@@ -5,6 +5,8 @@
import math
import omni.isaac.orbit.sim as sim_utils
from omni.isaac.orbit.assets import ArticulationCfg, AssetBaseCfg
from omni.isaac.orbit.envs import RLTaskEnvCfg
from omni.isaac.orbit.managers import ObservationGroupCfg as ObsGroup
from omni.isaac.orbit.managers import ObservationTermCfg as ObsTerm
......@@ -12,11 +14,46 @@ from omni.isaac.orbit.managers import RandomizationTermCfg as RandTerm
from omni.isaac.orbit.managers import RewardTermCfg as RewTerm
from omni.isaac.orbit.managers import SceneEntityCfg
from omni.isaac.orbit.managers import TerminationTermCfg as DoneTerm
from omni.isaac.orbit.scene import InteractiveSceneCfg
from omni.isaac.orbit.utils import configclass
import omni.isaac.orbit_tasks.classic.cartpole.mdp as mdp
from .cartpole_scene import CartpoleSceneCfg
##
# Pre-defined configs
##
from omni.isaac.orbit.assets.config.cartpole import CARTPOLE_CFG # isort:skip
##
# Scene definition
##
@configclass
class CartpoleSceneCfg(InteractiveSceneCfg):
"""Configuration for a cart-pole scene."""
# ground plane
ground = AssetBaseCfg(
prim_path="/World/ground",
spawn=sim_utils.GroundPlaneCfg(size=(100.0, 100.0)),
)
# cartpole
robot: ArticulationCfg = CARTPOLE_CFG.replace(prim_path="{ENV_REGEX_NS}/Robot")
# lights
dome_light = AssetBaseCfg(
prim_path="/World/DomeLight",
spawn=sim_utils.DomeLightCfg(color=(0.9, 0.9, 0.9), intensity=500.0),
)
distant_light = AssetBaseCfg(
prim_path="/World/DistantLight",
spawn=sim_utils.DistantLightCfg(color=(0.9, 0.9, 0.9), intensity=2500.0),
init_state=AssetBaseCfg.InitialStateCfg(rot=(0.738, 0.477, 0.477, 0.0)),
)
##
# MDP settings
......
# Copyright (c) 2022-2023, The ORBIT Project Developers.
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause
from __future__ import annotations
import omni.isaac.orbit.sim as sim_utils
from omni.isaac.orbit.assets import ArticulationCfg, AssetBaseCfg
from omni.isaac.orbit.assets.config.cartpole import CARTPOLE_CFG
from omni.isaac.orbit.scene import InteractiveSceneCfg
from omni.isaac.orbit.utils import configclass
@configclass
class CartpoleSceneCfg(InteractiveSceneCfg):
"""Configuration for a cartpole scene."""
# ground plane
ground = AssetBaseCfg(
prim_path="/World/ground",
spawn=sim_utils.GroundPlaneCfg(size=(100.0, 100.0)),
)
# cartpole
robot: ArticulationCfg = CARTPOLE_CFG.replace(prim_path="{ENV_REGEX_NS}/Robot")
# lights
dome_light = AssetBaseCfg(
prim_path="/World/DomeLight",
spawn=sim_utils.DomeLightCfg(color=(0.9, 0.9, 0.9), intensity=500.0),
)
distant_light = AssetBaseCfg(
prim_path="/World/DistantLight",
spawn=sim_utils.DistantLightCfg(color=(0.9, 0.9, 0.9), intensity=2500.0),
init_state=AssetBaseCfg.InitialStateCfg(rot=(0.738, 0.477, 0.477, 0.0)),
)
......@@ -3,9 +3,13 @@
#
# SPDX-License-Identifier: BSD-3-Clause
"""
This script demonstrates how to use the scene interface to quickly setup a scene with multiple
articulated robots and sensors.
"""This script demonstrates how to use the interactive scene interface to quickly setup a scene with multiple prims.
.. code-block:: bash
# Usage
./orbit.sh -p source/standalone/tutorials/03_scene/scene_creation.py --num_envs 32
"""
from __future__ import annotations
......@@ -20,7 +24,6 @@ from omni.isaac.orbit.app import AppLauncher
# add argparse arguments
parser = argparse.ArgumentParser(description="This script demonstrates how to use the scene interface.")
parser.add_argument("--num_envs", type=int, default=2, help="Number of environments to spawn.")
# append AppLauncher cli args
AppLauncher.add_app_launcher_args(parser)
# parse the arguments
......@@ -31,72 +34,95 @@ app_launcher = AppLauncher(args_cli)
simulation_app = app_launcher.app
"""Rest everything follows."""
import math
import torch
import traceback
import carb
import omni.isaac.orbit.sim as sim_utils
from omni.isaac.orbit.scene import InteractiveScene
from omni.isaac.orbit.assets import ArticulationCfg, AssetBaseCfg
from omni.isaac.orbit.assets.config import CARTPOLE_CFG
from omni.isaac.orbit.scene import InteractiveScene, InteractiveSceneCfg
from omni.isaac.orbit.sim import SimulationContext
from omni.isaac.orbit.utils import configclass
from omni.isaac.orbit_tasks.classic.cartpole.cartpole_scene import CartpoleSceneCfg
@configclass
class CartpoleSceneCfg(InteractiveSceneCfg):
"""Configuration for a cart-pole scene."""
# Main
def main():
# Load kit helper
sim = SimulationContext(sim_utils.SimulationCfg(device="cpu", use_gpu_pipeline=False))
# Set main camera
sim.set_camera_view([2.5, 2.5, 4.5], [0.0, 0.0, 2.0])
# ground plane
ground = AssetBaseCfg(prim_path="/World/defaultGroundPlane", spawn=sim_utils.GroundPlaneCfg())
# Spawn things into stage
scene = InteractiveScene(CartpoleSceneCfg(num_envs=args_cli.num_envs, env_spacing=5.0))
# lights
dome_light = AssetBaseCfg(
prim_path="/World/Light", spawn=sim_utils.DomeLightCfg(intensity=3000.0, color=(0.75, 0.75, 0.75))
)
# Play the simulator
sim.reset()
# articulation
cartpole: ArticulationCfg = CARTPOLE_CFG.replace(prim_path="{ENV_REGEX_NS}/Robot")
# Now we are ready!
print("[INFO]: Setup complete...")
def run_simulator(sim: sim_utils.SimulationContext, scene: InteractiveScene):
"""Runs the simulation loop."""
# Extract scene entities
# note: we only do this here for readability.
robot = scene["cartpole"]
# Define simulation stepping
sim_dt = sim.get_physics_dt()
count = 0
# Extract cartpole from InteractiveScene
cartpole = scene.articulations["robot"]
# Simulation loop
while simulation_app.is_running():
# reset
if count % 1000 == 0:
# Reset
if count % 500 == 0:
# reset counter
count = 0
# Get default joint positions and velocities and set them as targets
joint_pos, joint_vel = cartpole.data.default_joint_pos, cartpole.data.default_joint_vel
joint_ids, _ = cartpole.find_joints("cart_to_pole")
# Set joint position to be pi/8 so the pole will move
joint_pos[:, joint_ids[0]] = math.pi / 8.0
cartpole.set_joint_position_target(joint_pos)
cartpole.set_joint_velocity_target(joint_vel)
scene.write_data_to_sim()
# reset the scene entities
# root state
# we offset the root state by the origin since the states are written in simulation world frame
# if this is not done, then the robots will be spawned at the (0, 0, 0) of the simulation world
root_state = robot.data.default_root_state.clone()
root_state[:, :3] += scene.env_origins
robot.write_root_state_to_sim(root_state)
# set joint positions with some noise
joint_pos, joint_vel = robot.data.default_joint_pos.clone(), robot.data.default_joint_vel.clone()
joint_pos += torch.rand_like(joint_pos) * 0.1
robot.write_joint_state_to_sim(joint_pos, joint_vel)
# clear internal buffers
scene.reset()
print("[INFO]: Resetting robot state...")
# perform step
# Apply random action
# -- generate random joint efforts
efforts = torch.randn_like(robot.data.joint_pos) * 5.0
# -- apply action to the robot
robot.set_joint_effort_target(efforts)
# -- write data to sim
scene.write_data_to_sim()
# Perform step
sim.step()
# Increment counter
count += 1
# update buffers
# Update buffers
scene.update(sim_dt)
# End simulation loop
def main():
"""Main function."""
# Load kit helper
sim_cfg = sim_utils.SimulationCfg(device="cpu", use_gpu_pipeline=False)
sim = SimulationContext(sim_cfg)
# Set main camera
sim.set_camera_view([2.5, 0.0, 4.0], [0.0, 0.0, 2.0])
# Design scene
scene_cfg = CartpoleSceneCfg(num_envs=args_cli.num_envs, env_spacing=2.0)
scene = InteractiveScene(scene_cfg)
# Play the simulator
sim.reset()
# Now we are ready!
print("[INFO]: Setup complete...")
# Run the simulator
run_simulator(sim, scene)
if __name__ == "__main__":
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment