LMOps icon indicating copy to clipboard operation
LMOps copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

训练过程中CUDA out of memory

Open Yjonben opened this issue 7 months ago • 10 comments

我在4张A100上使用4卡模型并行训练，student是llama3-8b，teacher是llama3-70b，使用ds_config_zero2_offload运行成功时4张A100的GPU占用为47g/80g，在训练过程中会出现CUDA out of memory，请问如何解决这一问题 image

Yjonben avatar Jul 03 '24 01:07 Yjonben