Unverified Commit 5f7b3fa6 authored by Kelly Guo's avatar Kelly Guo Committed by GitHub

Moves segmentation workaround to base Camera class (#2243)

# Description

Moves segmentation workaround to base Camera class as the bug on
instanceable assets was also affecting the non-tiled cameras.

## Type of change

<!-- As you go through the list, delete the ones that are not
applicable. -->

- Bug fix (non-breaking change which fixes an issue)


## Screenshots

Please attach before and after screenshots of the change if applicable.

<!--
Example:

| Before | After |
| ------ | ----- |
| _gif/png before_ | _gif/png after_ |

To upload images to a PR -- simply drag and drop an image while in edit
mode and it should upload the image directly. You can then paste that
source into the above before/after sections.
-->

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [ ] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

<!--
As you go through the checklist above, you can mark something as done by
putting an x character in it

For example,
- [x] I have done this task
- [ ] I have not done this task
-->
parent 4156249f
...@@ -274,10 +274,10 @@ Breaking Changes ...@@ -274,10 +274,10 @@ Breaking Changes
.. attention:: .. attention::
We have identified a breaking feature for semantic segmentation and instance segmentation when using We have identified a breaking feature for semantic segmentation and instance segmentation when using
``TiledCamera`` with instanceable assets. Since the Isaac Sim 4.5 / Isaac Lab 2.0 release, semantic and instance ``Camera`` and ``TiledCamera`` with instanceable assets. Since the Isaac Sim 4.5 / Isaac Lab 2.0 release, semantic and instance
segmentation outputs only render the first tile correctly and produces blank outputs for the remaining tiles. segmentation outputs only render the first tile correctly and produces blank outputs for the remaining tiles.
We will be introducing a workaround for this fix to remove scene instancing if semantic segmentation or instance We will be introducing a workaround for this fix to remove scene instancing if semantic segmentation or instance
segmentation is required for ``TiledCamera`` until we receive a proper fix from Omniverse as part of the next Isaac Sim release. segmentation is required for ``Camera`` and ``TiledCamera`` until we receive a proper fix from Omniverse as part of the next Isaac Sim release.
Migration Guide Migration Guide
--------------- ---------------
......
...@@ -104,7 +104,7 @@ Changed ...@@ -104,7 +104,7 @@ Changed
Fixed Fixed
^^^^^ ^^^^^
* Fixed issue in :class:`~isaaclab.sensors.TiledCamera` where segmentation outputs only display the first tile * Fixed issue in :class:`~isaaclab.sensors.TiledCamera` and :class:`~isaaclab.sensors.Camera` where segmentation outputs only display the first tile
when scene instancing is enabled. A workaround is added for now to disable instancing when segmentation when scene instancing is enabled. A workaround is added for now to disable instancing when segmentation
outputs are requested. outputs are requested.
......
...@@ -17,7 +17,8 @@ import isaacsim.core.utils.stage as stage_utils ...@@ -17,7 +17,8 @@ import isaacsim.core.utils.stage as stage_utils
import omni.kit.commands import omni.kit.commands
import omni.usd import omni.usd
from isaacsim.core.prims import XFormPrim from isaacsim.core.prims import XFormPrim
from pxr import UsdGeom from isaacsim.core.version import get_version
from pxr import Sdf, UsdGeom
import isaaclab.sim as sim_utils import isaaclab.sim as sim_utils
from isaaclab.utils import to_camel_case from isaaclab.utils import to_camel_case
...@@ -141,6 +142,22 @@ class Camera(SensorBase): ...@@ -141,6 +142,22 @@ class Camera(SensorBase):
# Create empty variables for storing output data # Create empty variables for storing output data
self._data = CameraData() self._data = CameraData()
# HACK: we need to disable instancing for semantic_segmentation and instance_segmentation_fast to work
isaac_sim_version = get_version()
# checks for Isaac Sim v4.5 as this issue exists there
if int(isaac_sim_version[2]) == 4 and int(isaac_sim_version[3]) == 5:
if "semantic_segmentation" in self.cfg.data_types or "instance_segmentation_fast" in self.cfg.data_types:
omni.log.warn(
"Isaac Sim 4.5 introduced a bug in Camera and TiledCamera when outputting instance and semantic"
" segmentation outputs for instanceable assets. As a workaround, the instanceable flag on assets"
" will be disabled in the current workflow and may lead to longer load times and increased memory"
" usage."
)
stage = omni.usd.get_context().get_stage()
with Sdf.ChangeBlock():
for prim in stage.Traverse():
prim.SetInstanceable(False)
def __del__(self): def __del__(self):
"""Unsubscribes from callbacks and detach from the replicator registry.""" """Unsubscribes from callbacks and detach from the replicator registry."""
# unsubscribe callbacks # unsubscribe callbacks
......
...@@ -17,7 +17,7 @@ import omni.usd ...@@ -17,7 +17,7 @@ import omni.usd
import warp as wp import warp as wp
from isaacsim.core.prims import XFormPrim from isaacsim.core.prims import XFormPrim
from isaacsim.core.version import get_version from isaacsim.core.version import get_version
from pxr import Sdf, UsdGeom from pxr import UsdGeom
from isaaclab.utils.warp.kernels import reshape_tiled_image from isaaclab.utils.warp.kernels import reshape_tiled_image
...@@ -93,21 +93,6 @@ class TiledCamera(Camera): ...@@ -93,21 +93,6 @@ class TiledCamera(Camera):
) )
super().__init__(cfg) super().__init__(cfg)
# HACK: we need to disable instancing for semantic_segmentation and instance_segmentation_fast to work
isaac_sim_version = get_version()
# checks for Isaac Sim v4.5 as this issue exists there
if int(isaac_sim_version[2]) == 4 and int(isaac_sim_version[3]) == 5:
if "semantic_segmentation" in self.cfg.data_types or "instance_segmentation_fast" in self.cfg.data_types:
omni.log.warn(
"Isaac Sim 4.5 introduced a bug in TiledCamera when outputting instance and semantic segmentation"
" outputs for instanceable assets. As a workaround, the instanceable flag on assets will be"
" disabled in the current workflow and may lead to longer load times and increased memory usage."
)
stage = omni.usd.get_context().get_stage()
with Sdf.ChangeBlock():
for prim in stage.Traverse():
prim.SetInstanceable(False)
def __del__(self): def __del__(self):
"""Unsubscribes from callbacks and detach from the replicator registry.""" """Unsubscribes from callbacks and detach from the replicator registry."""
# unsubscribe from callbacks # unsubscribe from callbacks
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment