Anima icon indicating copy to clipboard operation
Anima copied to clipboard

Published 20 hours ago • verified publisher icon lyogavin

外推训练时max-position-embeddings需要做调整吗？

Open ScottishFold007 opened this issue 6 months ago • 1 comments

在进行外推训练时，max-position-embeddings这个参数需要做调整吗？比如之前是4096，外推到8K的话，需不需要改成8192？

ScottishFold007 avatar Dec 11 '23 09:12 ScottishFold007

We can do it. Will add it.

lyogavin avatar Dec 13 '23 15:12 lyogavin