DeepSpeedExamples icon indicating copy to clipboard operation
DeepSpeedExamples copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

Domino + PP

Open XZQshiyu opened this issue 10 months ago • 0 comments

I’m excited about the recent introduction of Domino and its impressive TP optimization. When I was using deepspeed-domino to better overlap comm & comp in TP, I found domino use forward_backward_no_pipelining() in schedules.py. Is that mean I couldn't use domino(tp optimization) and pp together?

XZQshiyu avatar Jan 09 '25 07:01 XZQshiyu