Fixes environment tests and disables bad ones (#3413)

# Description

Some visuomotor environments cannot run with 32 environments due to
memory limitations. We disable those tests along with a few bad
environments that need to be fixed.


## Type of change

<!-- As you go through the list, delete the ones that are not
applicable. -->

- Bug fix (non-breaking change which fixes an issue)

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [ ] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

<!--
As you go through the checklist above, you can mark something as done by
putting an x character in it

For example,
- [x] I have done this task
- [ ] I have not done this task
-->

---------
Co-authored-by: Rebecca Zhang <rebeccaz@nvidia.com>

.github/workflows/build.yml

@@ -10,6 +10,7 @@ on:
     branches:
       - devel
       - main
+      - 'release/**'
 
 # Concurrency control to prevent parallel runs on the same PR
 concurrency:

source/isaaclab_tasks/test/env_test_utils.py

@@ -123,11 +123,13 @@ def _run_environments(
         "Isaac-Stack-Cube-Franka-IK-Rel-Blueprint-v0",
         "Isaac-Stack-Cube-Instance-Randomize-Franka-IK-Rel-v0",
         "Isaac-Stack-Cube-Instance-Randomize-Franka-v0",
-        "Isaac-Stack-Cube-Franka-IK-Rel-Visuomotor-v0",
-        "Isaac-Stack-Cube-Franka-IK-Rel-Visuomotor-Cosmos-v0",
     ]:
         return
 
+    # skip these environments as they cannot be run with 32 environments within reasonable VRAM
+    if "Visuomotor" in task_name and num_envs == 32:
+        return
+
     # skip automate environments as they require cuda installation
     if task_name in ["Isaac-AutoMate-Assembly-Direct-v0", "Isaac-AutoMate-Disassembly-Direct-v0"]:
         return

tools/test_settings.py

@@ -18,17 +18,20 @@ DEFAULT_TIMEOUT = 300
 PER_TEST_TIMEOUTS = {
     "test_articulation.py": 500,
     "test_stage_in_memory.py": 500,
-    "test_environments.py": 2000,  # This test runs through all the environments for 100 steps each
+    "test_environments.py": 2500,  # This test runs through all the environments for 100 steps each
     "test_environments_with_stage_in_memory.py": (
-        2000
+        2500
     ),  # Like the above, with stage in memory and with and without fabric cloning
-    "test_environment_determinism.py": 500,  # This test runs through many the environments for 100 steps each
+    "test_environment_determinism.py": 1000,  # This test runs through many the environments for 100 steps each
     "test_factory_environments.py": 1000,  # This test runs through Factory environments for 100 steps each
     "test_multi_agent_environments.py": 800,  # This test runs through multi-agent environments for 100 steps each
     "test_generate_dataset.py": 500,  # This test runs annotation for 10 demos and generation until one succeeds
-    "test_environments_training.py": 6000,
+    "test_environments_training.py": (
+        6000
+    ),  # This test runs through training for several environments and compares thresholds
     "test_simulation_render_config.py": 500,
     "test_operational_space.py": 500,
+    "test_non_headless_launch.py": 1000,  # This test launches the app in non-headless mode and starts simulation
 }
 """A dictionary of tests and their timeouts in seconds.