PaddleNLP icon indicating copy to clipboard operation
PaddleNLP copied to clipboard

Published 20 hours ago • verified publisher icon PaddlePaddle

[llm]support long sequence training

Open lugimzzz opened this issue 4 months ago • 3 comments

PR types

New features

PR changes

Others

Description

新增支持单机8卡 llama 128k训练 待解决问题：

  1. 如果fuse_fused_head_and_loss_fn，开启pp和开eval的时候loss异常需要排查

lugimzzz avatar Sep 26 '24 11:09 lugimzzz