• TheIndoorDad's avatar
    Adds time-based mdp (observation) functions (#2332) · dca126ac
    TheIndoorDad authored
    # Description
    
    As discussed in #2284, I was writing an observation function to pass the
    time remaining in an episode (in seconds) to an observation term in a
    Manager-based environment, and found this could not be done without
    modifying `ManagerBasedRLEnv` to initialize `episode_length_buf` before
    managers are loaded. Here is a summary of changes made:
    
    * Added initialization of `episode_length_buf` in
    :meth:`load_managers()` of :class:`~isaaclab.envs.ManagerBasedRLEnv` to
    make it available for use in mdp functions. Note: existing
    initialization in :meth:`__init__` left in place in case it is needed
    for other use cases. Potentially redundant? Assess.
    * Added :attr:`~isaaclab.envs.ManagerBasedRLEnv.curr_episode_length` to
    :class:`~isaaclab.envs.ManagerBasedRLEnv` which returns reshaped
    ``episode_length_buf`` so it is visible as an attribute in the
    documentation.
    * Added time observation functions to `~isaaclab.envs.mdp.observations`
    module, :func:`~isaaclab.envs.mdp.observations.current_time_s` and
    :func:`~isaaclab.envs.mdp.observations.remaining_time_s`.
    
    I'm not certain whether the documentation will be updated automatically
    or if there are further steps I need to take. When I build the
    documentation on my machine it is updated, but the outputs are ignored
    by git. Please let me know if there's anything else I need to do.
    
    I could also use some advice on tests (apologies in advance for my lack
    of experience here, my background is not in software dev). Locally I
    modified the `Isaac-Velocity-Rough-Anymal-C-v0` task to add the two new
    observation functions, and began to train a policy in rsl_rl using the
    provided `scripts/reinforcement_learning/rsl_rl/train.py` script, and
    both were available to be viewed and appeared to be working correctly. I
    tried to run the existing suite of unit tests but it gave me an error I
    don't understand (see below). I also started to create a new script
    similar to
    [`isaaclab/test/envs/check_manager_based_env_anymal_locomotion.py`](https://github.com/isaac-sim/IsaacLab/blob/7de6d6fef9424c95fc68dc767af67ffbe0da6832/source/isaaclab/test/envs/check_manager_based_env_anymal_locomotion.py)
    but that would have required a policy trained using the new observation
    functions (which I can produce, but wasn't sure if that would be
    worthwhile since it wouldn't be available to others).
    
    Output when running `./isaaclab.sh -t`:
    ```
    [INFO] Warm starting the simulation app before running tests.                                                                                                                                                                                                                                                                        
    ERROR:root:Error warm starting the app: b'2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 16 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 17 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 18 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 19 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 20 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 21 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 22 belongs to.\n2025-04-17 18:14:17 [429ms] [Error] [omni.platforminfo.plugin] failed to find the package that core 23 belongs to.\nMESA-INTEL: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0\n\n'
    ```
    
    Cheers.
    
    ## Type of change
    
    - New feature (non-breaking change which adds functionality)
    - This change requires a documentation update
    
    ## Checklist
    
    - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
    `./isaaclab.sh --format`
    - [x] I have made corresponding changes to the documentation (_maybe_?)
    - [x] My changes generate no new warnings
    - [ ] I have added tests that prove my fix is effective or that my
    feature works (please advise)
    - [x] I have updated the changelog and the corresponding version in the
    extension's `config/extension.toml` file
    - [x] I have added my name to the `CONTRIBUTORS.md` or my name already
    exists there
    
    ---------
    Signed-off-by: 's avatarTheIndoorDad <167908515+TheIndoorDad@users.noreply.github.com>
    Signed-off-by: 's avatarMayank Mittal <12863862+Mayankm96@users.noreply.github.com>
    Co-authored-by: 's avatarMayank Mittal <12863862+Mayankm96@users.noreply.github.com>
    dca126ac
CONTRIBUTORS.md 2.22 KB