marella
falcon.cpp: tensor 'lm_head.weight' is missing from model
Whenever I try to load a QLoRA-merged Falcon 40B model, the error below happens.
error loading model: falcon.cpp: tensor 'lm_head.weight' is missing from model
The hacky way I did to resolve this error is to replace the line at
https://github.com/marella/ctransformers/blob/main/models/ggml/libfalcon.cpp#L1665
with
model.lm_head = model.tok_embeddings;
Is it possible to fix this issue so that it does not always expect to load lm_head.weight from the Falcon40B model?
I don't think it is correct to change it. Which scripts are you using for converting? Can you please try running the model using https://github.com/cmp-nct/ggllm.cpp and see if throws the same error.
I think it is actually a known issue. It is fixed in text-generation-inference at https://github.com/huggingface/text-generation-inference/pull/501#issuecomment-1663841922 with PR https://github.com/huggingface/text-generation-inference/pull/762/files#diff-2111bae5f77d998a3fe39888906b3c7be122313241ed6b69b0b0baf5abb735bbL57.