huggingface
SmolVLA Gym Simulation - Release?
Hello,
I've trained the smolvla_base for 200K steps. I'm trying to do a inference and visualize like we do for aloha or pusht. Could anyone guide me on this.
I dont have a robot arm, so Gym simulation is something I'm looking for, when will it be released?
I tried to use the lerobot/aloha_sim_insertion_human dataset to train smolvla_base and simulate it in the aloha Gym simulation. However, the inference effect is poor. I'm not sure if this attempt is feasible. Any reply would be helpful!
@zttiannnn The performance is poor. Could it be related to differences between the svla_so100_stacking dataset configs like state and the aloha one? From my understanding, the outreach by so-100 arm is very low compared to aloha , would it have any effects?
Tbh, I'm new to this, would like to hear your thoughts.
And i found in Twitter a fellow enthusiast posted that with koch Arm ,things are working fine. Not sure in a simulated env though.
https://x.com/XingdongZ/status/1930168794099659147
To be honest, I'm also new to this, but I don't think the dataset configs should be a crucial issue. 🤔 Thanks for your sharing. I'll go and consult him.
I tried to use the lerobot/aloha_sim_insertion_human dataset to train smolvla_base and simulate it in the aloha Gym simulation. However, the inference effect is poor. I'm not sure if this attempt is feasible. Any reply would be helpful!
Hi,have you ever found the simulation inference for SmolVLA? Actually I only find the DiffusionPolicy inference o(╥﹏╥)o
I think you're right. I checked the pretraining dataset, and most of the robots are so100. I didn't even see aloha. So it may not be possible to use the simulation environment to infer and view the model's performance.
I think you're right. I checked the pretraining dataset, and most of the robots are so100. I didn't even see aloha. So it may not be possible to use the simulation environment to infer and view the model's performance.
actually I even dont know how to inference so100 in sim env, there is no smolVLA inference for sim env right?
@zttiannnn
I tried to use the lerobot/aloha_sim_insertion_human dataset to train smolvla_base and simulate it in the aloha Gym simulation. However, the inference effect is poor. I'm not sure if this attempt is feasible. Any reply would be helpful!
Can you share the script that you have tried.
Also, the modelling part of smolvla quite a time uses adapt_to_pi_aloha
https://github.com/huggingface/lerobot/blob/main/lerobot/common/policies/smolvla/modeling_smolvla.py
I just tried train.py and eval.py in lerobot/scripts. For example, python lerobot/scripts/train.py --policy.path=lerobot/smolvla_base --dataset.repo_id=lerobot/aloha_sim_insertion_human --batch_size=64 --steps=20000. When running eval.py, select the aloha environment: --env.type=aloha
I just tried train.py and eval.py in lerobot/scripts. For example, python lerobot/scripts/train.py --policy.path=lerobot/smolvla_base --dataset.repo_id=lerobot/aloha_sim_insertion_human --batch_size=64 --steps=20000. When running eval.py, select the aloha environment: --env.type=aloha have you erver tried shorter chunk size? like 20?
Hey,
Do watch LeRobot hackathon solutions by various teams, they are supercool.
https://x.com/_raghuvamsi/status/1934280009122009115
- https://x.com/suessmannn/status/1934361919755813208
P.S> This msg is not relevant to this thread's topic.
smolvla said it has libero and metaworld, does these eval code open source? https://arxiv.org/abs/2506.01844
I am tring pi0 on the aloha, it has no performance too...
https://github.com/user-attachments/assets/f37bbd7a-6a1a-41ef-ae50-52e66c604dff
We have released both simulation environments in LeRobot used in smolvla. Meta-world and Libero https://huggingface.co/docs/lerobot/en/libero Enjoy! The checkpoint for smolvla libero is here: https://huggingface.co/HuggingFaceVLA/smolvla_libero
