Adds time-based mdp (observation) functions (#2332)

# Description

As discussed in #2284, I was writing an observation function to pass the
time remaining in an episode (in seconds) to an observation term in a
Manager-based environment, and found this could not be done without
modifying `ManagerBasedRLEnv` to initialize `episode_length_buf` before
managers are loaded. Here is a summary of changes made:

* Added initialization of `episode_length_buf` in
:meth:`load_managers()` of :class:`~isaaclab.envs.ManagerBasedRLEnv` to
make it available for use in mdp functions. Note: existing
initialization in :meth:`__init__` left in place in case it is needed
for other use cases. Potentially redundant? Assess.
* Added :attr:`~isaaclab.envs.ManagerBasedRLEnv.curr_episode_length` to
:class:`~isaaclab.envs.ManagerBasedRLEnv` which returns reshaped
``episode_length_buf`` so it is visible as an attribute in the
documentation.
* Added time observation functions to `~isaaclab.envs.mdp.observations`
module, :func:`~isaaclab.envs.mdp.observations.current_time_s` and
:func:`~isaaclab.envs.mdp.observations.remaining_time_s`.

I'm not certain whether the documentation will be updated automatically
or if there are further steps I need to take. When I build the
documentation on my machine it is updated, but the outputs are ignored
by git. Please let me know if there's anything else I need to do.

I could also use some advice on tests (apologies in advance for my lack
of experience here, my background is not in software dev). Locally I
modified the `Isaac-Velocity-Rough-Anymal-C-v0` task to add the two new
observation functions, and began to train a policy in rsl_rl using the
provided `scripts/reinforcement_learning/rsl_rl/train.py` script, and
both were available to be viewed and appeared to be working correctly. I
tried to run the existing suite of unit tests but it gave me an error I
don't understand (see below). I also started to create a new script
similar to
[`isaaclab/test/envs/check_manager_based_env_anymal_locomotion.py`](https://github.com/isaac-sim/IsaacLab/blob/7de6d6fef9424c95fc68dc767af67ffbe0da6832/source/isaaclab/test/envs/check_manager_based_env_anymal_locomotion.py)
but that would have required a policy trained using the new observation
functions (which I can produce, but wasn't sure if that would be
worthwhile since it wouldn't be available to others).

Output when running `./isaaclab.sh -t`:
```
[INFO] Warm starting the simulation app before running tests.                                                                                                                                                                                                                                                                        
ERROR:root:Error warm starting the app: b'2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 16 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 17 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 18 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 19 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 20 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 21 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 22 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 23 belongs to.\nMESA-INTEL: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0\n\n'
```

Cheers.

## Type of change

- New feature (non-breaking change which adds functionality)
- This change requires a documentation update

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [x] I have made corresponding changes to the documentation (_maybe_?)
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works (please advise)
- [x] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

---------
Signed-off-by: TheIndoorDad <167908515+TheIndoorDad@users.noreply.github.com>
Signed-off-by: Mayank Mittal <12863862+Mayankm96@users.noreply.github.com>
Co-authored-by: Mayank Mittal <12863862+Mayankm96@users.noreply.github.com>

CONTRIBUTORS.md

@@ -103,6 +103,7 @@ Guidelines for modifications:
 * Ryley McCarroll
 * Shafeef Omar
 * Shundo Kishi
+* Stefan Van de Mosselaer
 * Stephan Pleines
 * Tyler Lum
 * Victor Khaustov

source/isaaclab/config/extension.toml

 [package]
 
 # Note: Semantic Versioning is used: https://semver.org/
-version = "0.40.0"
+version = "0.40.1"
 
 
 # Description

source/isaaclab/docs/CHANGELOG.rst

 Changelog
 ---------
 
+0.40.1 (2025-06-02)
+~~~~~~~~~~~~~~~~~~~
+
+Added
+^^^^^
+* Added time observation functions to ~isaaclab.envs.mdp.observations module,
+  :func:`~isaaclab.envs.mdp.observations.current_time_s` and :func:`~isaaclab.envs.mdp.observations.remaining_time_s`.
+
+Changed
+^^^^^^^
+
+* Moved initialization of ``episode_length_buf`` outside of :meth:`load_managers()` of :class:`~isaaclab.envs.ManagerBasedRLEnv`
+  to make it available for mdp functions.
+
+
 0.40.0 (2025-05-16)
 ~~~~~~~~~~~~~~~~~~~
 

source/isaaclab/isaaclab/envs/manager_based_rl_env.py

@@ -75,14 +75,15 @@ class ManagerBasedRLEnv(ManagerBasedEnv, gym.Env):
         # -- counter for curriculum
         self.common_step_counter = 0
 
+        # initialize the episode length buffer BEFORE loading the managers to use it in mdp functions.
+        self.episode_length_buf = torch.zeros(cfg.scene.num_envs, device=cfg.sim.device, dtype=torch.long)
+
         # initialize the base class to setup the scene.
         super().__init__(cfg=cfg)
         # store the render mode
         self.render_mode = render_mode
 
         # initialize data and constants
-        # -- init buffers
-        self.episode_length_buf = torch.zeros(self.num_envs, device=self.device, dtype=torch.long)
         # -- set the framerate of the gym video recorder wrapper so that the playback speed of the produced video matches the simulation
         self.metadata["render_fps"] = 1 / self.step_dt
 

source/isaaclab/isaaclab/envs/mdp/observations.py

@@ -598,3 +598,18 @@ Commands.
 def generated_commands(env: ManagerBasedRLEnv, command_name: str) -> torch.Tensor:
     """The generated command from command term in the command manager with the given name."""
     return env.command_manager.get_command(command_name)
+
+
+"""
+Time.
+"""
+
+
+def current_time_s(env: ManagerBasedRLEnv) -> torch.Tensor:
+    """The current time in the episode (in seconds)."""
+    return env.episode_length_buf.unsqueeze(1) * env.step_dt
+
+
+def remaining_time_s(env: ManagerBasedRLEnv) -> torch.Tensor:
+    """The maximum time remaining in the episode (in seconds)."""
+    return env.max_episode_length_s - env.episode_length_buf.unsqueeze(1) * env.step_dt