Removes DLAA workaround for frame offset issue when rendering (#175)

Previously, small resolution renders that used DLSS for antialiasing
would sometimes display frame offset issues where the rendered frame is
not in sync with the current frame. To workaround this issue, we forced
the antialiasing mode to DLAA for small resolutions in both the Camera
and TiledCamra classes.

With recent changes in Isaac Sim 4.5, the frame offset issue no longer
appears, so this PR removes the workaround we've put in to avoid this
problem.

<!-- As you go through the list, delete the ones that are not
applicable. -->

- Bug fix (non-breaking change which fixes an issue)

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [ ] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

<!--
As you go through the checklist above, you can mark something as done by
putting an x character in it

For example,
- [x] I have done this task
- [ ] I have not done this task
-->

docs/source/overview/sensors/camera.rst

@@ -161,34 +161,7 @@ The info ``idToLabels`` dictionary will be the mapping from color to USD prim pa
 Current Limitations
 -------------------
 
-Due to current limitations in the renderer, we can have only **one** :class:`~sensors.TiledCamera` instance in the scene.
-For use cases that require a setup with more than one camera, we can imitate the multi-camera behavior by moving the location
-of the camera in between render calls in a step.
-
-For example, in a stereo vision setup, the below snippet can be implemented:
-
-.. code-block:: python
-
-    # render image from "first" camera
-    camera_data_1 = self._tiled_camera.data.output["rgb"].clone() / 255.0
-    # update camera transform to the "second" camera location
-    self._tiled_camera.set_world_poses(
-        positions=pos,
-        orientations=rot,
-        convention="world"
-    )
-    # step the renderer
-    self.sim.render()
-    self._tiled_camera.update(0, force_recompute=True)
-    # render image from "second" camera
-    camera_data_2 = self._tiled_camera.data.output["rgb"].clone() / 255.0
-
-Note that this approach still limits the rendering resolution to be identical for all cameras. Currently, there is no workaround
-to achieve different resolution images using :class:`~sensors.TiledCamera`. The best approach is to use the largest resolution out of all of the
-desired resolutions and add additional scaling or cropping operations to the rendered output as a post-processing step.
-
-In addition, there may be visible quality differences when comparing render outputs of different numbers of environments.
-Currently, any combined resolution that has a width less than 265 pixels or height less than 265 will automatically switch
-to the DLAA anti-aliasing mode, which does not perform up-sampling during anti-aliasing. For resolutions larger than 265 in both
-width and height dimensions, we default to using the "performance" DLSS mode for anti-aliasing for performance benefits.
-Anti-aliasing modes and other rendering parameters can be specified in the :class:`~sim.RenderCfg`.
+For performance reasons, we default to using the "performance" DLSS mode for anti-aliasing.
+This may result in renders of lower quality, which may be especially evident at lower resolutions.
+For higher quality rendering at a cost of performance, we recommend switching to the DLAA mode for anti-aliasing.
+Anti-aliasing modes and other rendering parameters can be specified in :class:`~sim.RenderCfg`.

source/extensions/omni.isaac.lab/docs/CHANGELOG.rst

@@ -163,7 +163,7 @@ Added
 * Added full buffer property to :class:`omni.isaac.lab.utils.buffers.circular_buffer.CircularBuffer`
 
 
-0.27.34 (2024-12-15)
+0.27.35 (2024-12-15)
 ~~~~~~~~~~~~~~~~~~~~
 
 Added
@@ -172,7 +172,7 @@ Added
 * Added action clip to all :class:`omni.isaac.lab.envs.mdp.actions`.
 
 
-0.27.33 (2024-12-14)
+0.27.34 (2024-12-14)
 ~~~~~~~~~~~~~~~~~~~~
 
 Changed
@@ -181,7 +181,7 @@ Changed
 * Added check for error below threshold in state machines to ensure the state has been reached.
 
 
-0.27.32 (2024-12-13)
+0.27.33 (2024-12-13)
 ~~~~~~~~~~~~~~~~~~~~
 
 Fixed
@@ -190,7 +190,7 @@ Fixed
 * Fixed the shape of ``quat_w`` in the ``apply_actions`` method of :attr:`~omni.isaac.lab.env.mdp.NonHolonomicAction` (previously (N,B,4), now (N,4) since the number of root bodies B is required to be 1). Previously ``apply_actions`` errored because ``euler_xyz_from_quat`` requires inputs of shape (N,4).
 
 
-0.27.31 (2024-12-11)
+0.27.32 (2024-12-11)
 ~~~~~~~~~~~~~~~~~~~~
 
 Changed
@@ -201,7 +201,7 @@ Changed
 * Improved documentation to clarify the usage of the :meth:`~omni.isaac.lab.envs.mdp.rewards.base_height_l2` function in both flat and rough terrain settings.
 
 
-0.27.30 (2024-12-11)
+0.27.31 (2024-12-11)
 ~~~~~~~~~~~~~~~~~~~~
 
 Fixed
@@ -211,7 +211,7 @@ Fixed
   Jacobian computed w.r.t. to the root frame of the robot. This helps ensure that root pose does not affect the tracking.
 
 
-0.27.29 (2024-12-09)
+0.27.30 (2024-12-09)
 ~~~~~~~~~~~~~~~~~~~~
 
 Fixed
@@ -221,7 +221,7 @@ Fixed
   return only the states of the specified environment IDs.
 
 
-0.27.28 (2024-12-06)
+0.27.29 (2024-12-06)
 ~~~~~~~~~~~~~~~~~~~~
 
 Fixed
@@ -231,7 +231,7 @@ Fixed
   :attr:`~omni.isaac.lab.assets.Articulation.root_physx_view` level.
 
 
-0.27.27 (2024-12-06)
+0.27.28 (2024-12-06)
 ~~~~~~~~~~~~~~~~~~~~
 
 Changed
@@ -242,7 +242,7 @@ Changed
   disabled. Using an articulation root for rigid bodies is not needed and decreases overall performance.
 
 
-0.27.26 (2024-12-06)
+0.27.27 (2024-12-06)
 ~~~~~~~~~~~~~~~~~~~~
 
 Fixed
@@ -252,7 +252,7 @@ Fixed
   Earlier, the projection names used snakecase instead of camelcase.
 
 
-0.27.25 (2024-12-06)
+0.27.26 (2024-12-06)
 ~~~~~~~~~~~~~~~~~~~~
 
 Added
@@ -270,7 +270,7 @@ Changed
   :class:`~omni.isaac.lab.sensors.Camera` did not clip them and had a different behavior for both types.
 
 
-0.27.24 (2024-12-05)
+0.27.25 (2024-12-05)
 ~~~~~~~~~~~~~~~~~~~~
 
 Fixed
@@ -279,6 +279,17 @@ Fixed
 * Fixed the condition in ``isaaclab.sh`` that checks whether ``pre-commit`` is installed before attempting installation.
 
 
+0.27.24 (2024-12-05)
+~~~~~~~~~~~~~~~~~~~~
+
+Fixed
+^^^^^
+
+* Removed workaround in :class:`omni.isaac.lab.sensors.TiledCamera` and :class:`omni.isaac.lab.sensors.Camera`
+  that was previously required to prevent frame offsets in renders. The denoiser setting is no longer
+  automatically modified based on the resolution of the cameras.
+
+
 0.27.23 (2024-12-04)
 ~~~~~~~~~~~~~~~~~~~~
 

source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/camera/camera.py

@@ -407,10 +407,6 @@ class Camera(SensorBase):
                 f" the number of environments ({self._num_envs})."
             )
 
-        # WAR: use DLAA antialiasing to avoid frame offset issue at small resolutions
-        if self.cfg.width < 265 or self.cfg.height < 265:
-            rep.settings.set_render_rtx_realtime(antialiasing="DLAA")
-
         # Create all env_ids buffer
         self._ALL_INDICES = torch.arange(self._view.count, device=self._device, dtype=torch.long)
         # Create frame count buffer

source/extensions/omni.isaac.lab/omni/isaac/lab/sensors/camera/tiled_camera.py

@@ -191,10 +191,6 @@ class TiledCamera(Camera):
         )
         self._render_product_paths = [rp.path]
 
-        # WAR: use DLAA antialiasing to avoid frame offset issue at small resolutions
-        if self._tiling_grid_shape()[0] * self.cfg.width < 265 or self._tiling_grid_shape()[1] * self.cfg.height < 265:
-            rep.settings.set_render_rtx_realtime(antialiasing="DLAA")
-
         # Define the annotators based on requested data types
         self._annotators = dict()
         for annotator_type in self.cfg.data_types:

source/extensions/omni.isaac.lab/test/sensors/test_tiled_camera.py

@@ -1424,6 +1424,89 @@ class TestTiledCamera(unittest.TestCase):
         # print info
         print(sensor)
 
+    def test_frame_offset_small_resolution(self):
+        """Test frame offset issue with small resolution camera."""
+        # Create sensor
+        camera_cfg = copy.deepcopy(self.camera_cfg)
+        camera_cfg.height = 80
+        camera_cfg.width = 80
+        tiled_camera = TiledCamera(camera_cfg)
+        # play sim
+        self.sim.reset()
+        # simulate some steps first to make sure objects are settled
+        for i in range(100):
+            # step simulation
+            self.sim.step()
+            # update camera
+            tiled_camera.update(self.dt)
+        # collect image data
+        image_before = tiled_camera.data.output["rgb"].clone() / 255.0
+
+        # update scene
+        stage = stage_utils.get_current_stage()
+        for i in range(10):
+            prim = stage.GetPrimAtPath(f"/World/Objects/Obj_{i:02d}")
+            color = Gf.Vec3f(0, 0, 0)
+            UsdGeom.Gprim(prim).GetDisplayColorAttr().Set([color])
+
+        # update rendering
+        self.sim.step()
+        # update camera
+        tiled_camera.update(self.dt)
+
+        # make sure the image is different
+        image_after = tiled_camera.data.output["rgb"].clone() / 255.0
+
+        # check difference is above threshold
+        self.assertGreater(
+            torch.abs((image_after - image_before).mean()), 0.05
+        )  # images of same color should be below 0.001
+
+    def test_frame_offset_large_resolution(self):
+        """Test frame offset issue with large resolution camera."""
+        # Create sensor
+        camera_cfg = copy.deepcopy(self.camera_cfg)
+        camera_cfg.height = 480
+        camera_cfg.width = 480
+        tiled_camera = TiledCamera(camera_cfg)
+
+        # modify scene to be less stochastic
+        stage = stage_utils.get_current_stage()
+        for i in range(10):
+            prim = stage.GetPrimAtPath(f"/World/Objects/Obj_{i:02d}")
+            color = Gf.Vec3f(1, 1, 1)
+            UsdGeom.Gprim(prim).GetDisplayColorAttr().Set([color])
+
+        # play sim
+        self.sim.reset()
+        # simulate some steps first to make sure objects are settled
+        for i in range(100):
+            # step simulation
+            self.sim.step()
+            # update camera
+            tiled_camera.update(self.dt)
+        # collect image data
+        image_before = tiled_camera.data.output["rgb"].clone() / 255.0
+
+        # update scene
+        for i in range(10):
+            prim = stage.GetPrimAtPath(f"/World/Objects/Obj_{i:02d}")
+            color = Gf.Vec3f(0, 0, 0)
+            UsdGeom.Gprim(prim).GetDisplayColorAttr().Set([color])
+
+        # update rendering
+        self.sim.step()
+        # update camera
+        tiled_camera.update(self.dt)
+
+        # make sure the image is different
+        image_after = tiled_camera.data.output["rgb"].clone() / 255.0
+
+        # check difference is above threshold
+        self.assertGreater(
+            torch.abs((image_after - image_before).mean()), 0.05
+        )  # images of same color should be below 0.001
+
     """
     Helper functions.
     """