Unverified Commit 5f7b3fa6 authored by Kelly Guo's avatar Kelly Guo Committed by GitHub

Moves segmentation workaround to base Camera class (#2243)

# Description

Moves segmentation workaround to base Camera class as the bug on
instanceable assets was also affecting the non-tiled cameras.

## Type of change

<!-- As you go through the list, delete the ones that are not
applicable. -->

- Bug fix (non-breaking change which fixes an issue)


## Screenshots

Please attach before and after screenshots of the change if applicable.

<!--
Example:

| Before | After |
| ------ | ----- |
| _gif/png before_ | _gif/png after_ |

To upload images to a PR -- simply drag and drop an image while in edit
mode and it should upload the image directly. You can then paste that
source into the above before/after sections.
-->

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [ ] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

<!--
As you go through the checklist above, you can mark something as done by
putting an x character in it

For example,
- [x] I have done this task
- [ ] I have not done this task
-->
parent 4156249f
......@@ -274,10 +274,10 @@ Breaking Changes
.. attention::
We have identified a breaking feature for semantic segmentation and instance segmentation when using
``TiledCamera`` with instanceable assets. Since the Isaac Sim 4.5 / Isaac Lab 2.0 release, semantic and instance
``Camera`` and ``TiledCamera`` with instanceable assets. Since the Isaac Sim 4.5 / Isaac Lab 2.0 release, semantic and instance
segmentation outputs only render the first tile correctly and produces blank outputs for the remaining tiles.
We will be introducing a workaround for this fix to remove scene instancing if semantic segmentation or instance
segmentation is required for ``TiledCamera`` until we receive a proper fix from Omniverse as part of the next Isaac Sim release.
segmentation is required for ``Camera`` and ``TiledCamera`` until we receive a proper fix from Omniverse as part of the next Isaac Sim release.
Migration Guide
---------------
......
......@@ -104,7 +104,7 @@ Changed
Fixed
^^^^^
* Fixed issue in :class:`~isaaclab.sensors.TiledCamera` where segmentation outputs only display the first tile
* Fixed issue in :class:`~isaaclab.sensors.TiledCamera` and :class:`~isaaclab.sensors.Camera` where segmentation outputs only display the first tile
when scene instancing is enabled. A workaround is added for now to disable instancing when segmentation
outputs are requested.
......
......@@ -17,7 +17,8 @@ import isaacsim.core.utils.stage as stage_utils
import omni.kit.commands
import omni.usd
from isaacsim.core.prims import XFormPrim
from pxr import UsdGeom
from isaacsim.core.version import get_version
from pxr import Sdf, UsdGeom
import isaaclab.sim as sim_utils
from isaaclab.utils import to_camel_case
......@@ -141,6 +142,22 @@ class Camera(SensorBase):
# Create empty variables for storing output data
self._data = CameraData()
# HACK: we need to disable instancing for semantic_segmentation and instance_segmentation_fast to work
isaac_sim_version = get_version()
# checks for Isaac Sim v4.5 as this issue exists there
if int(isaac_sim_version[2]) == 4 and int(isaac_sim_version[3]) == 5:
if "semantic_segmentation" in self.cfg.data_types or "instance_segmentation_fast" in self.cfg.data_types:
omni.log.warn(
"Isaac Sim 4.5 introduced a bug in Camera and TiledCamera when outputting instance and semantic"
" segmentation outputs for instanceable assets. As a workaround, the instanceable flag on assets"
" will be disabled in the current workflow and may lead to longer load times and increased memory"
" usage."
)
stage = omni.usd.get_context().get_stage()
with Sdf.ChangeBlock():
for prim in stage.Traverse():
prim.SetInstanceable(False)
def __del__(self):
"""Unsubscribes from callbacks and detach from the replicator registry."""
# unsubscribe callbacks
......
......@@ -17,7 +17,7 @@ import omni.usd
import warp as wp
from isaacsim.core.prims import XFormPrim
from isaacsim.core.version import get_version
from pxr import Sdf, UsdGeom
from pxr import UsdGeom
from isaaclab.utils.warp.kernels import reshape_tiled_image
......@@ -93,21 +93,6 @@ class TiledCamera(Camera):
)
super().__init__(cfg)
# HACK: we need to disable instancing for semantic_segmentation and instance_segmentation_fast to work
isaac_sim_version = get_version()
# checks for Isaac Sim v4.5 as this issue exists there
if int(isaac_sim_version[2]) == 4 and int(isaac_sim_version[3]) == 5:
if "semantic_segmentation" in self.cfg.data_types or "instance_segmentation_fast" in self.cfg.data_types:
omni.log.warn(
"Isaac Sim 4.5 introduced a bug in TiledCamera when outputting instance and semantic segmentation"
" outputs for instanceable assets. As a workaround, the instanceable flag on assets will be"
" disabled in the current workflow and may lead to longer load times and increased memory usage."
)
stage = omni.usd.get_context().get_stage()
with Sdf.ChangeBlock():
for prim in stage.Traverse():
prim.SetInstanceable(False)
def __del__(self):
"""Unsubscribes from callbacks and detach from the replicator registry."""
# unsubscribe from callbacks
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment