• ooctipus's avatar
    Supports composite observation space with proper min max in manager based env (#2811) · 69914828
    ooctipus authored
    This PR supports enables ManagerBasedRLEnv to properly support
    gymnaisum's composite observation gym.spaces.Dict at term-level with
    proper min, max specification. The benefit is that this will give
    rl-library a clearer signs how to pre-process the obseravtion data.
    
    Before:
    All terms are assigned with `gym.Spaces.Box(min=-np.inf, max=np.inf)`,
    one problem with this is that from rl-library side, is that some
    rl-libraries network construction depends heavily on observations space
    details. RL-library (e.g. sb3) looks at `gym.Spaces.Box.min` and
    `gym.Spaces.Box.max` to determine if they need to normalize it at
    library side.
    
    After:
    this PR utilizes on obs_terms's clip to determine if gym.Spaces.Box
    should be stricter. For example, environment obs_term returning
    gym.Spaces.Box(min=0, max=255) will be automatically detected by sb3,
    and a proper scalling be applied automatically. whereas
    gym.Spaces.Box(min=-inf, max=inf) will fail.
    
    while no special case is treated for gym.Spaces.Box(min=-1, max=1), but
    this will makes the rl-library easy to figure out that this term is
    already clipped, not more transformation needed.
    
    <!-- As you go through the list, delete the ones that are not
    applicable. -->
    
    - New feature (non-breaking change which adds functionality)
    
    Please attach before and after screenshots of the change if applicable.
    
    <!--
    Example:
    
    | Before | After |
    | ------ | ----- |
    | _gif/png before_ | _gif/png after_ |
    
    To upload images to a PR -- simply drag and drop an image while in edit
    mode and it should upload the image directly. You can then paste that
    source into the above before/after sections.
    -->
    
    - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
    `./isaaclab.sh --format`
    - [ ] I have made corresponding changes to the documentation
    - [x] My changes generate no new warnings
    - [x] I have added tests that prove my fix is effective or that my
    feature works
    - [x] I have updated the changelog and the corresponding version in the
    extension's `config/extension.toml` file
    - [x] I have added my name to the `CONTRIBUTORS.md` or my name already
    exists there
    
    <!--
    As you go through the checklist above, you can mark something as done by
    putting an x character in it
    
    For example,
    - [x] I have done this task
    - [ ] I have not done this task
    -->
    Signed-off-by: 's avatarKelly Guo <kellyg@nvidia.com>
    Co-authored-by: 's avatarKelly Guo <kellyg@nvidia.com>
    69914828
Name
Last commit
Last update
..
app Loading commit data...
assets Loading commit data...
controllers Loading commit data...
deps Loading commit data...
devices Loading commit data...
envs Loading commit data...
managers Loading commit data...
markers Loading commit data...
performance Loading commit data...
scene Loading commit data...
sensors Loading commit data...
sim Loading commit data...
terrains Loading commit data...
utils Loading commit data...