magicoder icon indicating copy to clipboard operation
magicoder copied to clipboard
verified publisher icon ise-uiuc

Huge performace drop when using DeepSpeed-ZeRO2 on 4 A40 GPUs.

Open twelfth-star opened this issue 1 year ago • 1 comments

Thank you for your impressive work!

I tried to use 4 A40 GPUs (48GB each) to reproduce your results. Due to the memory limit, I applied DeepSpeed-ZeRO2. However, the performance on APPS and livecodebench dropped significantly. Do you have any suggestions?

twelfth-star avatar Oct 13 '24 18:10 twelfth-star

Thank you for your impressive work!

I tried to use 4 A40 GPUs (48GB each) to reproduce your results. Due to the memory limit, I applied DeepSpeed-ZeRO2. However, the performance on APPS and livecodebench dropped significantly. Do you have any suggestions?

I'm encountering almost the same issue. Have you found a solution yet? Could it be caused by DeepSpeed?

wanglingxiang0717 avatar Apr 10 '25 10:04 wanglingxiang0717