• ooctipus's avatar
    Supports composite observation space with proper min max in manager based env (#2811) · 1f0be3d2
    ooctipus authored
    # Description
    This PR supports enables ManagerBasedRLEnv to properly support
    gymnaisum's composite observation gym.spaces.Dict at term-level with
    proper min, max specification. The benefit is that this will give
    rl-library a clearer signs how to pre-process the obseravtion data.
    
    Before:
    All terms are assigned with `gym.Spaces.Box(min=-np.inf, max=np.inf)`,
    one problem with this is that from rl-library side, is that some
    rl-libraries network construction depends heavily on observations space
    details. RL-library (e.g. sb3) looks at `gym.Spaces.Box.min` and
    `gym.Spaces.Box.max` to determine if they need to normalize it at
    library side.
    
    After:
    this PR utilizes on obs_terms's clip to determine if gym.Spaces.Box
    should be stricter. For example, environment obs_term returning
    gym.Spaces.Box(min=0, max=255) will be automatically detected by sb3,
    and a proper scalling be applied automatically. whereas
    gym.Spaces.Box(min=-inf, max=inf) will fail.
    
    while no special case is treated for gym.Spaces.Box(min=-1, max=1), but
    this will makes the rl-library easy to figure out that this term is
    already clipped, not more transformation needed.
    
    
    ## Type of change
    
    <!-- As you go through the list, delete the ones that are not
    applicable. -->
    
    - New feature (non-breaking change which adds functionality)
    
    ## Screenshots
    
    Please attach before and after screenshots of the change if applicable.
    
    <!--
    Example:
    
    | Before | After |
    | ------ | ----- |
    | _gif/png before_ | _gif/png after_ |
    
    To upload images to a PR -- simply drag and drop an image while in edit
    mode and it should upload the image directly. You can then paste that
    source into the above before/after sections.
    -->
    
    ## Checklist
    
    - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
    `./isaaclab.sh --format`
    - [ ] I have made corresponding changes to the documentation
    - [x] My changes generate no new warnings
    - [x] I have added tests that prove my fix is effective or that my
    feature works
    - [x] I have updated the changelog and the corresponding version in the
    extension's `config/extension.toml` file
    - [x] I have added my name to the `CONTRIBUTORS.md` or my name already
    exists there
    
    <!--
    As you go through the checklist above, you can mark something as done by
    putting an x character in it
    
    For example,
    - [x] I have done this task
    - [ ] I have not done this task
    -->
    Signed-off-by: 's avatarKelly Guo <kellyg@nvidia.com>
    Co-authored-by: 's avatarKelly Guo <kellyg@nvidia.com>
    1f0be3d2
Name
Last commit
Last update
..
config Loading commit data...
docs Loading commit data...
isaaclab Loading commit data...
test Loading commit data...
pyproject.toml Loading commit data...
setup.py Loading commit data...