pi0 inference example not working/not documented

[pablovela5620]

System Info

- `lerobot` version: 0.1.0
- Platform: Linux-6.8.0-52-generic-x86_64-with-glibc2.35
- Python version: 3.12.8
- Huggingface_hub version: 0.28.1
- Dataset version: 3.2.0
- Numpy version: 1.26.4
- PyTorch version (GPU?): 2.6.0+cu124 (True)
- Cuda version: 12040
- Using GPU in script?: yes

Information

• [x] One of the scripts in the examples/ folder of LeRobot
• [ ] My own task or dataset (give details below)

Reproduction

based on documentation provided here. I try running the pre-trained model but get the following error

python lerobot/scripts/eval.py --pretrained_policy.path=lerobot/pi0

usage: eval.py [-h] [--config_path str] [--env str] [--env.type {aloha,pusht,xarm}] [--env.task str] [--env.fps str] [--env.features str] [--env.features_map str]
               [--env.episode_length str] [--env.obs_type str] [--env.render_mode str] [--env.visualization_width str] [--env.visualization_height str] [--eval str]
               [--eval.n_episodes str] [--eval.batch_size str] [--eval.use_async_envs str] [--policy str] [--policy.type {act,diffusion,pi0,tdmpc,vqbet}]
               [--policy.replace_final_stride_with_dilation str] [--policy.pre_norm str] [--policy.dim_model str] [--policy.n_heads str] [--policy.dim_feedforward str]
               [--policy.feedforward_activation str] [--policy.n_encoder_layers str] [--policy.n_decoder_layers str] [--policy.use_vae str]
               [--policy.n_vae_encoder_layers str] [--policy.temporal_ensemble_coeff str] [--policy.kl_weight str] [--policy.optimizer_lr_backbone str]
               [--policy.drop_n_last_frames str] [--policy.use_separate_rgb_encoder_per_camera str] [--policy.down_dims str] [--policy.kernel_size str]
               [--policy.n_groups str] [--policy.diffusion_step_embed_dim str] [--policy.use_film_scale_modulation str] [--policy.noise_scheduler_type str]
               [--policy.num_train_timesteps str] [--policy.beta_schedule str] [--policy.beta_start str] [--policy.beta_end str] [--policy.prediction_type str]
               [--policy.clip_sample str] [--policy.clip_sample_range str] [--policy.num_inference_steps str] [--policy.do_mask_loss_for_padding str]
               [--policy.scheduler_name str] [--policy.chunk_size str] [--policy.max_state_dim str] [--policy.max_action_dim str]
               [--policy.resize_imgs_with_padding str] [--policy.empty_cameras str] [--policy.adapt_to_pi_aloha str] [--policy.use_delta_joint_actions_aloha str]
               [--policy.tokenizer_max_length str] [--policy.proj_width str] [--policy.num_steps str] [--policy.use_cache str] [--policy.attention_implementation str]
               [--policy.freeze_vision_encoder str] [--policy.train_expert_only str] [--policy.train_state_proj str] [--policy.scheduler_decay_steps str]
               [--policy.scheduler_decay_lr str] [--policy.n_action_repeats str] [--policy.horizon str] [--policy.n_action_steps str]
               [--policy.image_encoder_hidden_dim str] [--policy.state_encoder_hidden_dim str] [--policy.latent_dim str] [--policy.q_ensemble_size str]
               [--policy.mlp_dim str] [--policy.discount str] [--policy.use_mpc str] [--policy.cem_iterations str] [--policy.max_std str] [--policy.min_std str]
               [--policy.n_gaussian_samples str] [--policy.n_pi_samples str] [--policy.uncertainty_regularizer_coeff str] [--policy.n_elites str]
               [--policy.elite_weighting_temperature str] [--policy.gaussian_mean_momentum str] [--policy.max_random_shift_ratio str] [--policy.reward_coeff str]
               [--policy.expectile_weight str] [--policy.value_coeff str] [--policy.consistency_coeff str] [--policy.advantage_scaling str] [--policy.pi_coeff str]
               [--policy.temporal_decay_coeff str] [--policy.target_model_momentum str] [--policy.n_obs_steps str] [--policy.normalization_mapping str]
               [--policy.input_features str] [--policy.output_features str] [--policy.n_action_pred_token str] [--policy.action_chunk_size str]
               [--policy.vision_backbone str] [--policy.crop_shape str] [--policy.crop_is_random str] [--policy.pretrained_backbone_weights str]
               [--policy.use_group_norm str] [--policy.spatial_softmax_num_keypoints str] [--policy.n_vqvae_training_steps str] [--policy.vqvae_n_embed str]
               [--policy.vqvae_embedding_dim str] [--policy.vqvae_enc_hidden_dim str] [--policy.gpt_block_size str] [--policy.gpt_input_dim str]
               [--policy.gpt_output_dim str] [--policy.gpt_n_layer str] [--policy.gpt_n_head str] [--policy.gpt_hidden_dim str] [--policy.dropout str]
               [--policy.mlp_hidden_dim str] [--policy.offset_loss_weight str] [--policy.primary_code_loss_weight str] [--policy.secondary_code_loss_weight str]
               [--policy.bet_softmax_temperature str] [--policy.sequentially_select str] [--policy.optimizer_lr str] [--policy.optimizer_betas str]
               [--policy.optimizer_eps str] [--policy.optimizer_weight_decay str] [--policy.optimizer_vqvae_lr str] [--policy.optimizer_vqvae_weight_decay str]
               [--policy.scheduler_warmup_steps str] [--output_dir str] [--job_name str] [--device str] [--use_amp str] [--seed str]
eval.py: error: unrecognized arguments: --pretrained_policy.path=lerobot/pi0

I also tried

(lerobot) pablo@pablo-ubuntu:~/0Dev/repos/lerobot$ python lerobot/scripts/eval.py --policy.path=lerobot/pi0
Traceback (most recent call last):
  File "/home/pablo/0Dev/repos/lerobot/lerobot/scripts/eval.py", line 503, in <module>
    eval()
  File "/home/pablo/0Dev/repos/lerobot/.pixi/envs/default/lib/python3.12/site-packages/lerobot/configs/parser.py", line 119, in wrapper_inner
    cfg = draccus.parse(config_class=argtype, config_path=config_path, args=cli_args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pablo/0Dev/repos/lerobot/.pixi/envs/default/lib/python3.12/site-packages/draccus/argparsing.py", line 211, in parse
    return parser.parse_args(args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pablo/0Dev/repos/lerobot/.pixi/envs/default/lib/python3.12/site-packages/draccus/argparsing.py", line 102, in parse_args
    args, _ = self.parse_known_args(args, namespace, is_parse_args=True)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pablo/0Dev/repos/lerobot/.pixi/envs/default/lib/python3.12/site-packages/draccus/argparsing.py", line 136, in parse_known_args
    parsed_t = self._postprocessing(parsed_args)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pablo/0Dev/repos/lerobot/.pixi/envs/default/lib/python3.12/site-packages/draccus/argparsing.py", line 180, in _postprocessing
    cfg = decoding.decode(self.config_class, deflat_d)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pablo/0Dev/repos/lerobot/.pixi/envs/default/lib/python3.12/site-packages/draccus/parsers/registry_utils.py", line 78, in wrapper
    return base_func(*args, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pablo/0Dev/repos/lerobot/.pixi/envs/default/lib/python3.12/site-packages/draccus/parsers/decoding.py", line 48, in decode
    return get_decoding_fn(cls)(raw_value, ())  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pablo/0Dev/repos/lerobot/.pixi/envs/default/lib/python3.12/site-packages/draccus/parsers/decoding.py", line 157, in decode_dataclass
    raise DecodingError(path, f"Missing required field(s) {formatted_keys} for {stringify_type(cls)}")
draccus.utils.DecodingError: Missing required field(s) `env` for EvalPipelineConfig

but it did not work

Expected behavior

there should be clear documentation on how to run the pretrained model and an easy to use command that doesn't error

[aliberts]

It should be --policy.path= You'll also need to specify the environment with --eval.type= since it's a required argument (see EvalPipelineConfig) So for example

python lerobot/scripts/eval.py --policy.path=lerobot/pi0 --env.type=pusht

However right now this doesn't work because we're missing the train_config.json on that repo, we'll fix it soon. cc @Cadene

[pablovela5620]

It should be --policy.path= You'll also need to specify the environment with --eval.type= since it's a required argument (see EvalPipelineConfig) So for example

python lerobot/scripts/eval.py --policy.path=lerobot/pi0 --env.type=pusht However right now this doesn't work because we're missing the train_config.json on that repo, we'll fix it soon. cc @Cadene

Great! Thank you, looking forward to the fix

[IrvingF7]

Sorry to piggyback this, but another issue with pi0's example code is that, it says policy = Pi0Policy.from_pretrained("lerobot/pi0")

However, the policy in the source code is defined as PI0Policy.

[theophilegervet]

@aliberts I still get

FileNotFoundError: train_config.json not found on the HuggingFace Hub in lerobot/pi0

Do you have a working inference script for Pi0 in sim? cc @Cadene this would be super helpful!

@pablovela5620 Did you find a workaround?

[pablovela5620]

Sadly I have not, I think there's a pr to fix from @Cadene but I haven't seen any updates so far

[imstevenpmwork]

Hello @pablovela5620 , @theophilegervet and @IrvingF7,

Thanks for spotting this out! I'm currently working on a fix, it requires some refactoring but you can expect a PR in the following days 😄

[imstevenpmwork]

Sorry forgot to update the status of this issue, it was solved some weeks ago here: https://github.com/huggingface/lerobot/pull/812