the shape of params lm_head.wegiht is not compatible between base weight and delta weight

[sewellxie]

when running apply_delta.py of weight version v1.1, I find that the the shape of params lm_head.wegiht is [32000, 5120] in base weight, and is [32001, 5120] in delta weight. However, the shape of model.embed_tokends.weight are the same for these two weights.

Have anyone met this problem before?

[Ejafa]

use my llama Ejafa/llama_7B , I recovered successfully

[sewellxie]

use my llama Ejafa/llama_7B , I recovered successfully

Thanks. I solved this problem and recovered 13B model by concatenaing a torch.zeros([1, 5120]) to the end of llm_head.wegiht. But I still don't know why the shapes of lm_head.wegiht and model.embed_tokends.weight are not consistent.