Unverified Commit efa4e28b authored by Hongyu Li's avatar Hongyu Li Committed by GitHub

Fixes misalignment in motion vectors from TiledCamera (#2012)

# Description

I find the rendered motion_vectors (or optical flow) are not aligned
with other modalities (such as RGB, depth, segmentation, etc.).
This commit is a workaround to fix this misalignment issue.

Fixes #2003

<!-- As a practice, it is recommended to open an issue to have
discussions on the proposed pull request.
This makes it easier for the community to keep track of what is being
developed or added, and if a given feature
is demanded by more than one party. -->

## Type of change

<!-- As you go through the list, delete the ones that are not
applicable. -->

- Bug fix (non-breaking change which fixes an issue)

## Screenshots

Before:

![image](https://github.com/user-attachments/assets/acd2ba5c-a769-4759-b95b-1c31180355bb)

After:

![image](https://github.com/user-attachments/assets/a7c82707-9862-4c06-a83f-3950cdf7ba14)

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

---------
Signed-off-by: 's avatarHongyu Li <lihongyu0807@icloud.com>
Signed-off-by: 's avatarKelly Guo <kellyguo123@hotmail.com>
Signed-off-by: 's avatarMayank Mittal <12863862+Mayankm96@users.noreply.github.com>
Co-authored-by: 's avatarMayank Mittal <12863862+Mayankm96@users.noreply.github.com>
Co-authored-by: 's avatarKelly Guo <kellyguo123@hotmail.com>
parent f2ad2244
...@@ -55,6 +55,7 @@ Guidelines for modifications: ...@@ -55,6 +55,7 @@ Guidelines for modifications:
* Haoran Zhou * Haoran Zhou
* HoJin Jeon * HoJin Jeon
* Hongwei Xiong * Hongwei Xiong
* Hongyu Li
* Iretiayo Akinola * Iretiayo Akinola
* Jack Zeng * Jack Zeng
* Jan Kerner * Jan Kerner
......
[package] [package]
# Note: Semantic Versioning is used: https://semver.org/ # Note: Semantic Versioning is used: https://semver.org/
version = "0.34.7" version = "0.34.8"
# Description # Description
title = "Isaac Lab framework for Robot Learning" title = "Isaac Lab framework for Robot Learning"
......
Changelog Changelog
--------- ---------
0.34.8 (2025-03-04)
~~~~~~~~~~~~~~~~~~~
Fixed
^^^^^
* Fixed the issue of misalignment in the motion vectors from the :class:`TiledCamera`
with other modalities such as RGBA and depth.
0.34.7 (2025-03-04) 0.34.7 (2025-03-04)
~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~
......
...@@ -259,6 +259,11 @@ class TiledCamera(Camera): ...@@ -259,6 +259,11 @@ class TiledCamera(Camera):
ptr=tiled_data_buffer.ptr, shape=(*tiled_data_buffer.shape, 4), dtype=wp.uint8, device=self.device ptr=tiled_data_buffer.ptr, shape=(*tiled_data_buffer.shape, 4), dtype=wp.uint8, device=self.device
) )
# For motion vectors, we only require the first two channels of the tiled buffer
# Note: Not doing this breaks the alignment of the data (check: https://github.com/isaac-sim/IsaacLab/issues/2003)
if data_type == "motion_vectors":
tiled_data_buffer = tiled_data_buffer[:, :, :2].contiguous()
wp.launch( wp.launch(
kernel=reshape_tiled_image, kernel=reshape_tiled_image,
dim=(self._view.count, self.cfg.height, self.cfg.width), dim=(self._view.count, self.cfg.height, self.cfg.width),
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment