ChatGLM-6B icon indicating copy to clipboard operation
ChatGLM-6B copied to clipboard

Published 20 hours ago • verified publisher icon THUDM

现在全量训练为啥要设置fp16？这个精度不够，大模型容易导致不收敛，如何设置fp32或者bfp16？[Feature] <title>

Open TestNLP opened this issue 1 year ago • 1 comments

Is your feature request related to a problem? Please describe.

image

Solutions

无

Additional context

No response

TestNLP avatar Apr 24 '23 11:04 TestNLP

我记得是，不加fp16 默认用的fp32

cywjava avatar Apr 25 '23 04:04 cywjava

请问下您全量微调用的GPU显存多大呀？我们用了3块24GB的GPU，但是第一块GPU报了OOM，其余的GPU没满，请问您有遇到这种问题吗？

xiongxiaochu avatar May 23 '23 07:05 xiongxiaochu