chatglm_finetuning icon indicating copy to clipboard operation
chatglm_finetuning copied to clipboard

Published 20 hours ago • verified publisher icon ssbuild

deepspeed如何设置可以避免OOM

Open lianrzh opened this issue 1 year ago • 2 comments

请问，deepspeed可以如何设置参数支持模型全量微调，机器只有4张32g的V100，deepspeed里面提到32g最多可以支持13B模型的训练？默认deepspeed的配置使用的是stage2吗？

lianrzh avatar Apr 30 '23 07:04 lianrzh

+1

Forever-Young-l avatar May 04 '23 01:05 Forever-Young-l

+1

lxw0109 avatar May 10 '23 04:05 lxw0109