lerobot
lerobot copied to clipboard
Dataset stat miss empty_camera on pi0 when the VISUAL feature is not IDENTITY.
System Info
As described. I find there is a additional feature added to the feature list at the post processing step of pi0's configuration [here](https://github.com/huggingface/lerobot/blob/a6015a55f930cdc51fdb035d68533d1434b1cf43/lerobot/common/policies/pi0/configuration_pi0.py#L116).
It works fine for now because the VISUAL feature processing is now default to IDENTITY. But it creates error like below if you use different processing for VISUAL feature.
Traceback (most recent call last):
File "/home/jiaye.yang/.local/share/mamba/envs/il/lib/python3.11/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 92, in _call_target
return _target_(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jiaye.yang/.local/share/mamba/envs/il/lib/python3.11/site-packages/lerobot/common/policies/pi0/modeling_pi0.py", line 243, in __init__
self.normalize_inputs = Normalize(config.input_features, config.normalization_mapping, dataset_stats)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jiaye.yang/.local/share/mamba/envs/il/lib/python3.11/site-packages/lerobot/common/policies/normalize.py", line 147, in __init__
stats_buffers = create_stats_buffers(features, norm_map, stats)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jiaye.yang/.local/share/mamba/envs/il/lib/python3.11/site-packages/lerobot/common/policies/normalize.py", line 83, in create_stats_buffers
if isinstance(stats[key]["mean"], np.ndarray):
~~~~~^^^^^
KeyError: 'observation.images.empty_camera_0'
This is because there is no entry for these newly created feature in the dataset stats.
Information
- [ ] One of the scripts in the examples/ folder of LeRobot
- [ ] My own task or dataset (give details below)
Reproduction
try to run pi0 with VISUAL feature normalization mode not being identity and non-zero number of empty_cameras.
Expected behavior
it works without error