TencentPretrain icon indicating copy to clipboard operation
TencentPretrain copied to clipboard

Published 20 hours ago • verified publisher icon Tencent

关于扩充词表再增量预训练的疑问

Open qiancheng99 opened this issue 1 year ago • 0 comments

  1. Linly-Chinese-LLaMA-7B没有扩充词表。如果我自己训练了一个sentencepiece词表并和原版词表合并以后，要怎么改动代码扩充embedding进行预训练呢？
  2. 是否支持llama 13b的增量预训练呢？

qiancheng99 avatar May 19 '23 03:05 qiancheng99