Is there a first pretraining stage model for Llama 7b?

[johnmullan]

Or should we just carry on from the existing checkpoint?

Thanks

[slacklife]

I have the same question, and everyone can also take a look at this discussion

[terminator123]

i have the same question, have u resolved it ?

[zhangwengyu999]

Same question, I use the provided 7B MiniGPT-4 checkpoint for the second FT with Llama2, but it gives error size mismatch for llama_proj.weight: copying a param with shape torch.Size([4096, 768]) from checkpoint, the shape in current model is torch.Size([4096, 5632]). However, it works fine when I use the ckpt for second FT with Vicuna 7B, may be this ckpt is for Vicuna 7B only. Could you please provide the first stage ckpt that works for Llama2. Thank you very much!