Jack BAI comments

Results 37 comments of


                                            Jack BAI

[Bug]: Gemma-3 (27B) can't load save_pretrained() checkpoint: AssertionError: expected size 5376==2560, stride 1==1 at dim=0

Update: the error is still not resolved even if I use the **original config** downloaded from HF

[Bug]: Gemma-3 (27B) can't load save_pretrained() checkpoint: AssertionError: expected size 5376==2560, stride 1==1 at dim=0

> If this issue is still Open @BiEchi ? Yes, this problem is not resolved yet.

[Bug]: Gemma-3 (27B) can't load save_pretrained() checkpoint: AssertionError: expected size 5376==2560, stride 1==1 at dim=0

Did you pull the repo and compile yourself? If you're using the pip pre-built version, there might be some discrepancies.

[Bug]: Gemma-3 (27B) can't load save_pretrained() checkpoint: AssertionError: expected size 5376==2560, stride 1==1 at dim=0

Got it - @hmellor do you know anyone of the repo team that might be interested in this vllm bug?

[Bug]: Gemma-3 (27B) can't load save_pretrained() checkpoint: AssertionError: expected size 5376==2560, stride 1==1 at dim=0

I just got a minor update that if we pass enforce_eager to the V1 engine, then this problem will no longer remain, but the generation will get stuck at 80%...

[Bug]: Gemma-3 (27B) can't load save_pretrained() checkpoint: AssertionError: expected size 5376==2560, stride 1==1 at dim=0

@DarkLight1337 @WoosukKwon I also managed to run the saved model with huggingface. I believe this is a vLLM issue while loading the model. I suspect there's some differences between HF...

[Bug]: Gemma-3 (27B) can't load save_pretrained() checkpoint: AssertionError: expected size 5376==2560, stride 1==1 at dim=0

I am not observing this as of today for Gemma-3 with the latest version of vllm.