lerobot icon indicating copy to clipboard operation
lerobot copied to clipboard

Dataset stat miss empty_camera on pi0 when the VISUAL feature is not IDENTITY.

Open LumenYoung opened this issue 9 months ago • 0 comments

System Info

As described. I find there is a additional feature added to the feature list at the post processing step of pi0's configuration [here](https://github.com/huggingface/lerobot/blob/a6015a55f930cdc51fdb035d68533d1434b1cf43/lerobot/common/policies/pi0/configuration_pi0.py#L116).

It works fine for now because the VISUAL feature processing is now default to IDENTITY. But it creates error like below if you use different processing for VISUAL feature.


Traceback (most recent call last):
  File "/home/jiaye.yang/.local/share/mamba/envs/il/lib/python3.11/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 92, in _call_target
    return _target_(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jiaye.yang/.local/share/mamba/envs/il/lib/python3.11/site-packages/lerobot/common/policies/pi0/modeling_pi0.py", line 243, in __init__
    self.normalize_inputs = Normalize(config.input_features, config.normalization_mapping, dataset_stats)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jiaye.yang/.local/share/mamba/envs/il/lib/python3.11/site-packages/lerobot/common/policies/normalize.py", line 147, in __init__
    stats_buffers = create_stats_buffers(features, norm_map, stats)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jiaye.yang/.local/share/mamba/envs/il/lib/python3.11/site-packages/lerobot/common/policies/normalize.py", line 83, in create_stats_buffers
    if isinstance(stats[key]["mean"], np.ndarray):
                  ~~~~~^^^^^
KeyError: 'observation.images.empty_camera_0'


This is because there is no entry for these newly created feature in the dataset stats.

Information

  • [ ] One of the scripts in the examples/ folder of LeRobot
  • [ ] My own task or dataset (give details below)

Reproduction

try to run pi0 with VISUAL feature normalization mode not being identity and non-zero number of empty_cameras.

Expected behavior

it works without error

LumenYoung avatar Mar 24 '25 10:03 LumenYoung