• ooctipus's avatar
    Supports composite observation space with proper min max in manager based env (#2811) · 1f0be3d2
    ooctipus authored
    # Description
    This PR supports enables ManagerBasedRLEnv to properly support
    gymnaisum's composite observation gym.spaces.Dict at term-level with
    proper min, max specification. The benefit is that this will give
    rl-library a clearer signs how to pre-process the obseravtion data.
    
    Before:
    All terms are assigned with `gym.Spaces.Box(min=-np.inf, max=np.inf)`,
    one problem with this is that from rl-library side, is that some
    rl-libraries network construction depends heavily on observations space
    details. RL-library (e.g. sb3) looks at `gym.Spaces.Box.min` and
    `gym.Spaces.Box.max` to determine if they need to normalize it at
    library side.
    
    After:
    this PR utilizes on obs_terms's clip to determine if gym.Spaces.Box
    should be stricter. For example, environment obs_term returning
    gym.Spaces.Box(min=0, max=255) will be automatically detected by sb3,
    and a proper scalling be applied automatically. whereas
    gym.Spaces.Box(min=-inf, max=inf) will fail.
    
    while no special case is treated for gym.Spaces.Box(min=-1, max=1), but
    this will makes the rl-library easy to figure out that this term is
    already clipped, not more transformation needed.
    
    
    ## Type of change
    
    <!-- As you go through the list, delete the ones that are not
    applicable. -->
    
    - New feature (non-breaking change which adds functionality)
    
    ## Screenshots
    
    Please attach before and after screenshots of the change if applicable.
    
    <!--
    Example:
    
    | Before | After |
    | ------ | ----- |
    | _gif/png before_ | _gif/png after_ |
    
    To upload images to a PR -- simply drag and drop an image while in edit
    mode and it should upload the image directly. You can then paste that
    source into the above before/after sections.
    -->
    
    ## Checklist
    
    - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
    `./isaaclab.sh --format`
    - [ ] I have made corresponding changes to the documentation
    - [x] My changes generate no new warnings
    - [x] I have added tests that prove my fix is effective or that my
    feature works
    - [x] I have updated the changelog and the corresponding version in the
    extension's `config/extension.toml` file
    - [x] I have added my name to the `CONTRIBUTORS.md` or my name already
    exists there
    
    <!--
    As you go through the checklist above, you can mark something as done by
    putting an x character in it
    
    For example,
    - [x] I have done this task
    - [ ] I have not done this task
    -->
    Signed-off-by: 's avatarKelly Guo <kellyg@nvidia.com>
    Co-authored-by: 's avatarKelly Guo <kellyg@nvidia.com>
    1f0be3d2
Name
Last commit
Last update
..
isaaclab Loading commit data...
isaaclab_assets Loading commit data...
isaaclab_mimic Loading commit data...
isaaclab_rl Loading commit data...
isaaclab_tasks Loading commit data...