lerobot icon indicating copy to clipboard operation
lerobot copied to clipboard

rerun always begins playback at 30fps, even when the dataset prescribes a different fps.

Open RussTedrake opened this issue 1 year ago • 5 comments

System Info

- `lerobot` version: 0.1.0 (building from source: commit: 461d5472d329a6a461f56a85175d4744b8e78e61)
- Platform: macOS-14.5-arm64-arm-64bit
- Python version: 3.12.3
- Huggingface_hub version: 0.23.4
- Dataset version: 2.20.0
- Numpy version: 1.26.4
- PyTorch version (GPU?): 2.3.1 (False)
- Cuda version: N/A
- Using GPU in script?: no

Information

  • [X] One of the scripts in the examples/ folder of LeRobot
  • [ ] My own task or dataset (give details below)

Reproduction

python lerobot/scripts/visualize_dataset.py \
        --repo-id lerobot/xarm_push_medium \
        --episode-index 1

or with my dataset, which was the original motivation:

 python lerobot/scripts/visualize_dataset.py \
        --repo-id russtedrake/tri-small-BimanualStackPlatesOnTableFromTable \
        --episode-index 1        

Expected behavior

The rerun.io viewer should open, and start playing the data back at 15 fps (as specified in the dataset). But instead it always defaults to 30 fps playback.

For my dataset, I hoped for 10 fps playback, but still get 30 fps by default. I've also confirmed that my mp4s are encoded at the proper rate, and playback at the correct speed e.g. here: https://huggingface.co/datasets/RussTedrake/tri-small-BimanualStackPlatesOnTableFromTable/blob/main/videos/scene_left_0_episode_000002.mp4

I wonder if a call to this might help: https://docs.rs/re_viewer_context/0.16.1/re_viewer_context/struct.TimeControl.html#method.set_fps

fwiw -- i've also blown away my local cache and confirmed that the dataset which is getting loaded does indeed report dataset.fps does indeed equal 15 (or 10) respectively.

cc @notmahi who I was talking to about this last week.

RussTedrake avatar Jul 21 '24 23:07 RussTedrake