DeepSpeedExamples icon indicating copy to clipboard operation
DeepSpeedExamples copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

the memory usage of zero3 is larger than zero1

Open blldd opened this issue 1 year ago • 1 comments

When I run the step1_supervised_finetuning script, I find that the memory usage of zero3 is larger than that of zero1, which seems unreasonable. Is there any other optimization here?

blldd avatar Apr 13 '23 09:04 blldd

@blldd could you provide more details, like the training scripts, GPU numbers etc?

yaozhewei avatar Apr 13 '23 15:04 yaozhewei

Close the issue since there is no followup. Please reopen it if necessary

yaozhewei avatar Apr 24 '23 19:04 yaozhewei