xtuner icon indicating copy to clipboard operation
xtuner copied to clipboard

Published 20 hours ago • verified publisher icon InternLM

请问有支持PoSE的打算吗？

Open 2793145003 opened this issue 9 months ago • 0 comments

PoSE：https://arxiv.org/abs/2309.10400 https://github.com/OpenAccess-AI-Collective/axolotl/pull/1567 不需要全长度微调，无需修改 attention 好像可以用短上下文训练长上下文

2793145003 avatar May 06 '24 06:05 2793145003