Pi0 scratch training

[lromor]

Hi!

Awesome respository, very neatly organized. I have a question though, in the paper it is mentioned you did experiments training the model with the action expert trained from scratch. Is there a recipe to reproduce pi0 "scratch" training on a task instead of fine tuning?

In the paper, I assume the "scratch action expert" is randomly initialized while the rest of the ViT and language expert is from pretrained weights. Are the pre-trained weights from the original paligemma model or from pi0 base?

Thanks, -l