Cheng Qian

icon company University of Michigan icon location Ann Arbor, MI

Results 4 comments of Cheng Qian

预训练第一阶段，需要冻结原版LLaMA词表的embedding吗

请问训练整个词表是否会影响原本的embedding？冻结原版LLaMA的embedding会不会效果更好呢？请问您有没有做过这个实验，或者怎么看待这个问题呢？感谢您的解答！

Distinguish between bold and non-bold Fonts

Do you find any methods to extract these bold texts such as subtitles? Thank you!

预训练llama 13b报错 NotImplementedError: Cannot copy out of meta tensor; no data!

torch==1.13.1 transformers==4.28.1 还是会报这个错

Some weights of GPTNeoXForCausalLM were not initialized from the model checkpoint

I met exactly the same issue except that I am using llama. Did you find the correct way to load the trained rm model?