starcoder2 icon indicating copy to clipboard operation
starcoder2 copied to clipboard
verified publisher icon bigcode-project

"lm_head.weight" not in the parameters of Starcoder2-3B and Starcoder2-7B (Huggingface version)

Open renhouxing opened this issue 1 year ago • 2 comments

There is no lm_head.weight in the parameters of Starcoder2-3B and Starcoder2-7B. Is it because of tied embedding?

renhouxing avatar Mar 30 '24 03:03 renhouxing

Yes!

NouamaneTazi avatar Apr 15 '24 01:04 NouamaneTazi

ok, thanks for your response!

renhouxing avatar Apr 15 '24 05:04 renhouxing