Does Pi0 use PaliGemma VLM pretrained model weights?

[lxysl]

I attempted to finetune the Pi0 model, but noticed that it does not download the pretrained weights of Paligemma from Hugging Face. Specifically, I found that Pi0 initializes the VLM with:

self.paligemma = PaliGemmaForConditionalGeneration(config=config.paligemma_config)

instead of using:

AutoModel.from_pretrained("google/paligemma-3b-pt-224")

This seems to result in the model not loading the pretrained weights.

Could you please confirm whether this is the intended behavior? Should Pi0 load Paligemma’s pretrained weights from Hugging Face, or is there a reason it initializes the model from scratch?

Thank you!