PaddleNLP icon indicating copy to clipboard operation
PaddleNLP copied to clipboard

Published 20 hours ago • verified publisher icon PaddlePaddle

[fix] DP8 training

Open RuohengMa opened this issue 9 months ago • 2 comments

PR types

Bug fixes

PR changes

Others

Description

Llama2 13B 4k can not run in pure DP8 configuration on XPU. Fix this issue by setting no_sync() for model when running in pure DP configuration.

RuohengMa avatar May 20 '24 03:05 RuohengMa