Skip to content

K
KincoActuatorIsaacLab
Commit e4342c99 authored by Hunter Hansen's avatar Hunter Hansen Committed by Mayank Mittal
Browse files
Options

Adds the ability create composite compose yamls/.env files to container.py (#545) 

# Description

This PR recreates the capabilities initially made by @farbod-farshidian
[here](https://github.com/isaac-sim/IsaacLab/pull/455) for the new
Pythonized-container interface.

## Type of change

<!-- As you go through the list, delete the ones that are not
applicable. -->

- New feature (non-breaking change which adds functionality)
- This change requires a documentation update

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have run all the tests with `./isaaclab.sh --test` and they pass
- [ ] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

<!--
As you go through the checklist above, you can mark something as done by
putting an x character in it

For example,
- [x] I have done this task
- [ ] I have not done this task
-->

---------
Signed-off-by: 's avatarHunter Hansen <50837800+hhansen-bdai@users.noreply.github.com>
Signed-off-by: 's avatarJames Smith <142246516+jsmith-bdai@users.noreply.github.com>
Co-authored-by: 's avatarJames Smith <142246516+jsmith-bdai@users.noreply.github.com>
Co-authored-by: 's avatarDavid Hoeller <dhoeller@nvidia.com>
parent f565c33d
Hide whitespace changes
Inline Side-by-side
Showing with 137 additions and 49 deletions
docker/container.py
View file @ e4342c99
......@@ -17,16 +17,48 @@ def main():
parser = argparse.ArgumentParser(description="Utility for using Docker with Isaac Lab.")
subparsers = parser.add_subparsers(dest="command", required=True)
# We have to create a separate parent parser for common options to our subparsers
# We have to create separate parent parsers for common options to our subparsers
parent_parser = argparse.ArgumentParser(add_help=False)
parent_parser.add_argument("profile", nargs="?", default="base", help="Optional container profile specification.")
parent_parser.add_argument(
"--files",
nargs="*",
default=None,
help=(
"Allows additional .yaml files to be passed to the docker compose command. Files will be merged with"
" docker-compose.yaml in the order in which they are provided."
),
)
parent_parser.add_argument(
"--env-files",
nargs="*",
default=None,
help=(
"Allows additional .env files to be passed to the docker compose command. Files will be merged with"
" .env.base in the order in which they are provided."
),
)
# Actual command definition begins here
subparsers.add_parser(
"start", help="Build the docker image and create the container in detached mode.", parents=[parent_parser]
"start",
help="Build the docker image and create the container in detached mode.",
parents=[parent_parser],
)
subparsers.add_parser(
"enter", help="Begin a new bash process within an existing Isaac Lab container.", parents=[parent_parser]
)
config = subparsers.add_parser(
"config",
help=(
"Generate a docker-compose.yaml from the passed yamls, .envs, and either print to the terminal or create a"
" yaml at output_yaml"
),
parents=[parent_parser],
)
config.add_argument(
"--output-yaml", nargs="?", default=None, help="Yaml file to write config output to. Defaults to None."
)
subparsers.add_parser(
"copy", help="Copy build and logs artifacts from the container to the host machine.", parents=[parent_parser]
)
......@@ -38,11 +70,12 @@ def main():
raise RuntimeError("Docker is not installed! Please check the 'Docker Guide' for instruction.")
# Creating container interface
ci = IsaacLabContainerInterface(context_dir=Path(__file__).resolve().parent, profile=args.profile)
ci = IsaacLabContainerInterface(
context_dir=Path(__file__).resolve().parent, profile=args.profile, yamls=args.files, envs=args.env_files
)
print(f"[INFO] Using container profile: {ci.profile}")
if args.command == "start":
print(f"[INFO] Building the docker image and starting the container {ci.container_name} in the background...")
x11_outputs = x11_utils.x11_check(ci.statefile)
if x11_outputs is not None:
(x11_yaml, x11_envar) = x11_outputs
......@@ -50,15 +83,13 @@ def main():
ci.environ.update(x11_envar)
ci.start()
elif args.command == "enter":
print(f"[INFO] Entering the existing {ci.container_name} container in a bash session...")
x11_utils.x11_refresh(ci.statefile)
ci.enter()
elif args.command == "config":
ci.config(args.output_yaml)
elif args.command == "copy":
print(f"[INFO] Copying artifacts from the 'isaac-lab-{ci.container_name}' container...")
ci.copy()
print("\n[INFO] Finished copying the artifacts from the container.")
elif args.command == "stop":
print(f"[INFO] Stopping the launched docker container {ci.container_name}...")
ci.stop()
x11_utils.x11_cleanup(ci.statefile)
else:
......
docker/utils/isaaclab_container_interface.py
View file @ e4342c99
......@@ -18,7 +18,14 @@ class IsaacLabContainerInterface:
Interface for managing Isaac Lab containers.
"""
def __init__(self, context_dir: Path, profile: str = "base", statefile: None | Statefile = None):
def __init__(
self,
context_dir: Path,
profile: str = "base",
statefile: None | Statefile = None,
yamls: list[str] | None = None,
envs: list[str] | None = None,
):
"""
Initialize the IsaacLabContainerInterface with the given parameters.
......@@ -26,6 +33,8 @@ class IsaacLabContainerInterface:
context_dir : The context directory for Docker operations.
statefile : An instance of the Statefile class to manage state variables. If not provided, initializes a Statefile(path=self.context_dir/.container.yaml).
profile : The profile name for the container. Defaults to "base".
yamls : A list of yamls to extend docker-compose.yaml. They will be extended in the order they are provided.
envs : A list of envs to extend .env.base. They will be extended in the order they are provided.
"""
self.context_dir = context_dir
if statefile is None:
......@@ -41,12 +50,16 @@ class IsaacLabContainerInterface:
self.container_name = f"isaac-lab-{self.profile}"
self.image_name = f"isaac-lab-{self.profile}:latest"
self.environ = os.environ
self.resolve_image_extension()
self.resolve_image_extension(yamls, envs)
self.load_dot_vars()
def resolve_image_extension(self):
def resolve_image_extension(self, yamls: list[str] | None = None, envs: list[str] | None = None):
"""
Resolve the image extension by setting up YAML files, profiles, and environment files for the Docker compose command.
Args:
yamls (List[str], optional): A list of yamls to extend docker-compose.yaml. They will be extended in the order they are provided.
envs (List[str], optional): A list of envs to extend .env.base. They will be extended in the order they are provided.
"""
self.add_yamls = ["--file", "docker-compose.yaml"]
self.add_profiles = ["--profile", f"{self.profile}"]
......@@ -54,6 +67,14 @@ class IsaacLabContainerInterface:
if self.profile != "base":
self.add_env_files += ["--env-file", f".env.{self.profile}"]
if yamls is not None:
for yaml in yamls:
self.add_yamls += ["--file", yaml]
if envs is not None:
for env in envs:
self.add_env_files += ["--env-file", env]
def load_dot_vars(self):
"""
Load environment variables from .env files into a dictionary.
......@@ -104,6 +125,7 @@ class IsaacLabContainerInterface:
"""
Build and start the Docker container using the Docker compose command.
"""
print(f"[INFO] Building the docker image and starting the container {self.container_name} in the background...")
subprocess.run(
[
"docker",
......@@ -138,6 +160,7 @@ class IsaacLabContainerInterface:
RuntimeError: If the container is not running.
"""
if self.is_container_running():
print(f"[INFO] Entering the existing {self.container_name} container in a bash session...")
subprocess.run([
"docker",
"exec",
......@@ -159,11 +182,9 @@ class IsaacLabContainerInterface:
RuntimeError: If the container is not running.
"""
if self.is_container_running():
print(f"[INFO] Stopping the launched docker container {self.container_name}...")
subprocess.run(
["docker", "compose", "--file", "docker-compose.yaml"]
+ self.add_profiles
+ self.add_env_files
+ ["down"],
["docker", "compose"] + self.add_yamls + self.add_profiles + self.add_env_files + ["down"],
check=False,
cwd=self.context_dir,
env=self.environ,
......@@ -182,18 +203,21 @@ class IsaacLabContainerInterface:
RuntimeError: If the container is not running.
"""
if self.is_container_running():
print(f"[INFO] Copying artifacts from the 'isaac-lab-{self.container_name}' container...")
if output_dir is None:
output_dir = self.context_dir
output_dir = output_dir.joinpath("artifacts")
if not output_dir.exists():
if not output_dir.is_dir():
output_dir.mkdir()
artifacts = {
"logs": output_dir.joinpath("logs"),
"docs/_build": output_dir.joinpath("docs"),
"data_storage": output_dir.joinpath("data_storage"),
Path(self.dot_vars["DOCKER_ISAACLAB_PATH"]).joinpath("logs"): output_dir.joinpath("logs"),
Path(self.dot_vars["DOCKER_ISAACLAB_PATH"]).joinpath("docs/_build"): output_dir.joinpath("docs"),
Path(self.dot_vars["DOCKER_ISAACLAB_PATH"]).joinpath("data_storage"): output_dir.joinpath(
"data_storage"
),
}
for container_path, host_path in artifacts.items():
print(f"\t - /workspace/isaaclab/{container_path} -> {host_path}")
print(f"\t -{container_path} -> {host_path}")
for path in artifacts.values():
shutil.rmtree(path, ignore_errors=True)
for container_path, host_path in artifacts.items():
......@@ -201,10 +225,32 @@ class IsaacLabContainerInterface:
[
"docker",
"cp",
f"isaac-lab-{self.profile}:/workspace/isaaclab/{container_path}/",
f"isaac-lab-{self.profile}:{container_path}/",
f"{host_path}",
],
check=False,
)
print("\n[INFO] Finished copying the artifacts from the container.")
else:
raise RuntimeError(f"The container '{self.container_name}' is not running")
def config(self, output_yaml: Path | None = None) -> None:
"""
Generate a docker-compose.yaml from the passed yamls, .envs, and either print to the
terminal or create a yaml at output_yaml
Args:
output_yaml (Path, optional): The absolute path of the yaml file to write the output to, if any. Defaults
to None, and simply prints to the terminal
"""
print("[INFO] Configuring the passed options into a yaml...")
if output_yaml is not None:
output = ["--output", output_yaml]
else:
output = []
subprocess.run(
["docker", "compose"] + self.add_yamls + self.add_profiles + self.add_env_files + ["config"] + output,
check=False,
cwd=self.context_dir,
env=self.environ,
)
docs/source/deployment/cluster.rst
View file @ e4342c99
......@@ -58,7 +58,7 @@ the user cluster password from being requested multiple times.
Configuring the cluster parameters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
First, you need to configure the cluster-specific parameters in ``docker/.env.base`` file.
First, you need to configure the cluster-specific parameters in ``docker/cluster/.env.cluster`` file.
The following describes the parameters that need to be configured:
- ``CLUSTER_JOB_SCHEDULER``:
The job scheduler/workload manager used by your cluster. Currently, we support SLURM and
......@@ -82,6 +82,9 @@ The following describes the parameters that need to be configured:
- ``CLUSTER_PYTHON_EXECUTABLE``:
The path within Isaac Lab to the Python executable that should be executed in the submitted job.
When a ``job`` is submitted, it will also use variables defined in ``docker/.env.base``, though these
should be correct by default.
Exporting to singularity image
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
......@@ -94,10 +97,11 @@ To export to a singularity image, execute the following command:
.. code:: bash
./docker/container.sh push [profile]
./docker/cluster/cluster_interface.sh push [profile]
This command will create a singularity image under ``docker/exports`` directory and
upload it to the defined location on the cluster. Be aware that creating the singularity
upload it to the defined location on the cluster. It requires that you have previously
built the image with the ``container.py`` interface. Be aware that creating the singularity
image can take a while.
``[profile]`` is an optional argument that specifies the container profile to be used. If no profile is
specified, the default profile ``base`` will be used.
......@@ -105,7 +109,7 @@ specified, the default profile ``base`` will be used.
.. note::
By default, the singularity image is created without root access by providing the ``--fakeroot`` flag to
the ``apptainer build`` command. In case the image creation fails, you can try to create it with root
access by removing the flag in ``docker/container.sh``.
access by removing the flag in ``docker/cluster/cluster_interface.sh``.
Defining the job parameters
......@@ -168,7 +172,7 @@ To submit a job on the cluster, the following command can be used:
.. code:: bash
./docker/container.sh job [profile] "argument1" "argument2" ...
./docker/cluster/cluster_interface.sh job [profile] "argument1" "argument2" ...
This command will copy the latest changes in your code to the cluster and submit a job. Please ensure that
your Python executable's output is stored under ``isaaclab/logs`` as this directory will be copied again
......@@ -184,13 +188,13 @@ ANYmal rough terrain locomotion training can be executed with the following comm
.. code:: bash
./docker/container.sh job --task Isaac-Velocity-Rough-Anymal-C-v0 --headless --video --enable_cameras
./docker/cluster/cluster_interface.sh job --task Isaac-Velocity-Rough-Anymal-C-v0 --headless --video --enable_cameras
The above will, in addition, also render videos of the training progress and store them under ``isaaclab/logs`` directory.
.. note::
The ``./docker/container.sh job`` command will copy the latest changes in your code to the cluster. However,
The ``./docker/cluster/cluster_interface.sh job`` command will copy the latest changes in your code to the cluster. However,
it will not delete any files that have been deleted locally. These files will still exist on the cluster
which can lead to issues. In this case, we recommend removing the ``CLUSTER_ISAACLAB_DIR`` directory on
the cluster and re-run the command.
......
docs/source/deployment/docker.rst
View file @ e4342c99
......@@ -80,7 +80,7 @@ needed to run Isaac Lab inside a Docker container. A subset of these are summari
store frequently re-used resources compiled by Isaac Sim, such as shaders, and to retain logs, data, and documents.
* ``base.env``: Stores environment variables required for the ``base`` build process and the container itself. ``.env``
files which end with something else (i.e. ``.env.ros2``) define these for `image_extension <#isaac-lab-image-extensions>`_.
* ``container.py``: A script that interfaces with tools in ``isaaclab_container_utils`` to configure and build the image,
* ``container.py``: A script that interfaces with tools in ``utils`` to configure and build the image,
and run and interact with the container.
Running the Container
......@@ -106,9 +106,11 @@ or else they will default to image_extension ``base``:
1. ``start``: This builds the image and brings up the container in detached mode (i.e. in the background).
2. ``enter``: This begins a new bash process in an existing isaaclab container, and which can be exited
without bringing down the container.
3. ``copy``: This copies the ``logs``, ``data_storage`` and ``docs/_build`` artifacts, from the ``isaac-lab-logs``, ``isaac-lab-data`` and ``isaac-lab-docs``
3. ``config``: This outputs the compose.yaml which would be result from the inputs given to ``container.py start``. This command is useful
for debugging a compose configuration.
4. ``copy``: This copies the ``logs``, ``data_storage`` and ``docs/_build`` artifacts, from the ``isaac-lab-logs``, ``isaac-lab-data`` and ``isaac-lab-docs``
volumes respectively, to the ``docker/artifacts`` directory. These artifacts persist between docker container instances and are shared between image extensions.
4. ``stop``: This brings down the container and removes it.
5. ``stop``: This brings down the container and removes it.
The following shows how to launch the container in a detached state and enter it:
......@@ -117,6 +119,11 @@ The following shows how to launch the container in a detached state and enter it
# Launch the container in detached mode
# We don't pass an image extension arg, so it defaults to 'base'
python docker/container.py start
# If we want to add .env or .yaml files to customize our compose config,
# we can simply specify them in the same manner as the compose cli
# python docker/container.py start --file my-compose.yaml --env-file .env.my-vars
# Enter the container
# We pass 'base' explicitly, but if we hadn't it would default to 'base'
python docker/container.py enter base
......@@ -131,7 +138,7 @@ To copy files from the base container to the host machine, you can use the follo
The script ``container.py`` provides a wrapper around this command to copy the ``logs`` , ``data_storage`` and ``docs/_build``
directories to the ``docker/artifacts`` directory. This is useful for copying the logs, data and documentation:
.. code::
.. code:: bash
# stop the container
python docker/container.py stop
......@@ -143,9 +150,9 @@ X11 forwarding
The container supports X11 forwarding, which allows the user to run GUI applications from the container and display them
on the host machine.
The first time a container is started with ``./docker/container.sh start``, the script prompts
the user whether to activate X11 forwarding. This will create a file ``docker/.container.yaml`` to store the user's choice.
Subsequently, X11 forwarding can be toggled by changing ``__ISAACLAB_X11_FORWARDING_ENABLED`` to 0 or 1 in ``docker/.container.yaml``.
The first time a container is started with ``python docker/container.py start``, the script prompts
the user whether to activate X11 forwarding. This will create a file ``docker/.container.cfg`` to store the user's choice.
Subsequently, X11 forwarding can be toggled by changing ``__ISAACLAB_X11_FORWARDING_ENABLED`` to 0 or 1 in ``docker/.container.cfg``.
Python Interpreter
......@@ -197,7 +204,7 @@ Isaac Lab Image Extensions
The produced image depends upon the arguments passed to ``container.py start`` and ``container.py stop``. These
commands accept an ``image_extension`` as an additional argument. If no argument is passed, then these
commands default to ``base``. Currently, the only valid ``image_extension`` arguments are (``base``, ``ros2``).
Only one ``image_extension`` can be passed at a time, and the produced container will be named ``isaaclab``.
Only one ``image_extension`` can be passed at a time, and the produced container will be named ``isaac-lab-${profile}``.
.. code:: bash
......
docs/source/deployment/run_docker_example.rst
View file @ e4342c99
......@@ -3,7 +3,7 @@ Running an example with Docker
From the root of the ``Isaac Lab`` repository, the ``docker`` directory contains all the Docker relevant files. These include the three files
(**Dockerfile**, **docker-compose.yaml**, **.env**) which are used by Docker, and an additional script that we use to interface with them,
**container.sh**.
**container.py**.
In this tutorial, we will learn how to use the Isaac Lab Docker container for development. For a detailed description of the Docker setup,
including installation and obtaining access to an Isaac Sim image, please reference the :ref:`deployment-docker`. For a description
......@@ -18,26 +18,26 @@ To build the Isaac Lab container from the root of the Isaac Lab repository, we w
.. code-block:: console
./docker/container.sh start
python docker/container.py start
The terminal will first pull the base IsaacSim image, build the Isaac Lab image's additional layers on top of it, and run the Isaac Lab container.
This should take several minutes upon the first build but will be shorter in subsequent runs as Docker's caching prevents repeated work.
If we run the command ``docker container ls`` on the terminal, the output will list the containers that are running on the system. If
everything has been set up correctly, a container with the ``NAME`` **isaaclab** should appear, similar to below:
everything has been set up correctly, a container with the ``NAME`` **isaac-lab-base** should appear, similar to below:
.. code-block:: console
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
483d1d5e2def isaaclab "bash" 30 seconds ago Up 30 seconds isaaclab
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
483d1d5e2def isaac-lab-base "bash" 30 seconds ago Up 30 seconds isaac-lab-base
Once the container is up and running, we can enter it from our terminal.
.. code-block:: console
./docker/container.sh enter
python docker/container.py enter
On entering the Isaac Lab container, we are in the terminal as the superuser, ``root``. This environment contains a copy of the
......@@ -105,7 +105,7 @@ the following command to retrieve our logs from the Docker container and put the
.. code-block:: console
./container.sh copy
python container.py copy
We will see a terminal readout reporting the artifacts we have retrieved from the container. If we navigate to
......@@ -113,25 +113,25 @@ We will see a terminal readout reporting the artifacts we have retrieved from th
by the script above.
Each of the directories under ``artifacts`` corresponds to Docker `volumes`_ mapped to directories
within the container and the ``container.sh copy`` command copies them from those `volumes`_ to these directories.
within the container and the ``container.py copy`` command copies them from those `volumes`_ to these directories.
We could return to the Isaac Lab Docker terminal environment by running ``container.sh enter`` again,
We could return to the Isaac Lab Docker terminal environment by running ``container.py enter`` again,
but we have retrieved our logs and wish to go inspect them. We can stop the Isaac Lab Docker container with the following command:
.. code-block:: console
./container.sh stop
python container.py stop
This will bring down the Docker Isaac Lab container. The image will persist and remain available for further use, as will
the contents of any `volumes`_. If we wish to free up the disk space taken by the image, (~20.1GB), and do not mind repeating
the build process when we next run ``./container.sh start``, we may enter the following command to delete the **isaaclab** image:
the build process when we next run ``python container.py start``, we may enter the following command to delete the **isaac-lab-base** image:
.. code-block:: console
docker image rm isaaclab
docker image rm isaac-lab-base
A subsequent run of ``docker image ls`` will show that the image tagged **isaaclab** is now gone. We can repeat the process for the
A subsequent run of ``docker image ls`` will show that the image tagged **isaac-lab-base** is now gone. We can repeat the process for the
underlying NVIDIA container if we wish to free up more space. If a more powerful method of freeing resources from Docker is desired,
please consult the documentation for the `docker prune`_ commands.
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment