Commit 807d194a authored by peterd-NV's avatar peterd-NV Committed by Kelly Guo

Fixes Mimic annotation script hang & doc updates (#332)

# Description

The following bugs in Mimic are fixed:

1. Annotation script would hang during startup unless the Isaac Sim
window was clicked on when playing visuomotor env. Fixed by removing
unneccessary env full sim reset prior to reset during startup.
2. Add instructions to docs on where to modify docker compose file to
mount spacemouse.
3. Add to docs the command to run demo replay for GR1 env. 
4. Fix "Pink controller frametask" hyperlink in docs. 

## Type of change

<!-- As you go through the list, delete the ones that are not
applicable. -->

- Bug fix (non-breaking change which fixes an issue)
- Documentation update

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [ ] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

<!--
As you go through the checklist above, you can mark something as done by
putting an x character in it

For example,
- [x] I have done this task
- [ ] I have not done this task
-->
parent ddc901c5
......@@ -43,7 +43,11 @@ For smoother operation and off-axis operation, we recommend using a SpaceMouse a
where ``<#>`` is the device index of the connected SpaceMouse.
Only compatible with the SpaceMouse Wireless and SpaceMouse Compact models from 3Dconnexion.
If you are using the IsaacLab + CloudXR container deployment (:ref:`cloudxr-teleoperation`), you can add the ``devices`` attribute under the ``services -> isaac-lab-base`` section of the
``docker/docker-compose.cloudxr-runtime.patch.yaml`` file.
Isaac Lab is only compatible with the SpaceMouse Wireless and SpaceMouse Compact models from 3Dconnexion.
For tasks that benefit from the use of an extended reality (XR) device with hand tracking, Isaac Lab supports using NVIDIA CloudXR to immersively stream the scene to compatible XR devices for teleoperation.
......@@ -329,7 +333,7 @@ Optional: Collect and annotate demonstrations
The differential IK controller requires the user's wrist pose to be close to the robot's initial or current pose for optimal performance.
Rapid movements of the user's wrist may cause it to deviate significantly from the goal state, which could prevent the IK controller from finding the optimal solution.
This may result in a mismatch between the user's wrist and the robot's wrist.
You can increase the gain of the all `Pink-IK controller's FrameTasks<https://github.com/isaac-sim/IsaacLab-Internal/blob/devel/source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/pick_place/pickplace_gr1t2_env_cfg.py>`__ to track the AVP wrist poses with lower latency.
You can increase the gain of the all `Pink-IK controller's FrameTasks <https://github.com/isaac-sim/IsaacLab-Internal/blob/devel/source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/pick_place/pickplace_gr1t2_env_cfg.py>`__ to track the AVP wrist poses with lower latency.
However, this may lead to more jerky motion.
Separately, the finger joints of the robot are retargeted to the user's finger joints using the `dex-retargeting <https://github.com/dexsuite/dex-retargeting>`_ library.
......@@ -357,6 +361,19 @@ We recommend 10 successful demonstrations for good data generation results.
If a demo fails during data collection, the environment can be reset using the teleoperation controls panel in the XR teleop client
on the Apple Vision Pro or via voice control by saying "reset". See :ref:`teleoperate-apple-vision-pro` for more details.
You can replay the collected demonstrations by running the following command:
.. code:: bash
./isaaclab.sh -p scripts/tools/replay_demos.py \
--device cpu \
--task Isaac-PickPlace-GR1T2-Abs-v0 \
--dataset_file ./datasets/dataset_gr1.hdf5 --enable_pinocchio
.. note::
Non-determinism may be observed during replay as physics in IsaacLab are not determimnistically reproducible when using ``env.reset``.
Unlike the prior Franka stacking task, the GR-1 pick and place task uses manual annotation to define subtasks.
Each demo requires a single annotation between the first and second subtask of the right arm. This annotation ("S" button press) should be done when the right robot arm finishes the "idle" subtask and begins to
move towards the target object. Annotate the demonstrations by running the following command:
......
......@@ -220,8 +220,7 @@ def main():
# no need to annotate the last subtask term signal, so remove it from the list
subtask_term_signal_names[eef_name].pop()
# reset environment - we do a full simulation reset to achieve full determinism
env.sim.reset()
# reset environment
env.reset()
# Only enables inputs if this script is NOT headless mode
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment