Unverified Commit 45ee5317 authored by shauryadNv's avatar shauryadNv Committed by GitHub

Adds note to mimic cosmos pipeline doc for eval (#530)

# Description

Updated the Mimic-Cosmos pipeline doc to include a minor note on eval
script runtime as per QA suggestion.

## Type of change

- Bug fix (non-breaking change which fixes an issue)
- This change requires a documentation update

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there\
parent b1af9e3c
...@@ -388,6 +388,9 @@ Example usage for the cube stacking task: ...@@ -388,6 +388,9 @@ Example usage for the cube stacking task:
--num_rollouts 15 \ --num_rollouts 15 \
--rendering_mode performance --rendering_mode performance
.. note::
This script can take over a day or even longer to run (depending on the hardware being used). This behavior is expected.
We use the above script to compare models trained with 1000 Mimic-generated demonstrations, 2000 Mimic-generated demonstrations and 2000 Cosmos-Mimic-generated demonstrations (1000 original mimic + 1000 Cosmos augmented) respectively. We use the same seeds (0, 1000 and 5000) for all three models and provide the metrics (averaged across best checkpoints for each seed) below: We use the above script to compare models trained with 1000 Mimic-generated demonstrations, 2000 Mimic-generated demonstrations and 2000 Cosmos-Mimic-generated demonstrations (1000 original mimic + 1000 Cosmos augmented) respectively. We use the same seeds (0, 1000 and 5000) for all three models and provide the metrics (averaged across best checkpoints for each seed) below:
.. rubric:: Model Comparison .. rubric:: Model Comparison
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment