DeepSpeed icon indicating copy to clipboard operation
DeepSpeed copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

Flops Profiler

Open BhAem opened this issue 5 months ago • 1 comments

In deepspeed.utils.timer.py，what is the difference between the below items: BACKWARD_MICRO_TIMER = 'bwd_microstep' BACKWARD_GLOBAL_TIMER = 'bwd' BACKWARD_INNER_MICRO_TIMER = 'bwd_inner_microstep' BACKWARD_INNER_GLOBAL_TIMER = 'bwd_inner' BACKWARD_REDUCE_MICRO_TIMER = 'bwd_allreduce_microstep' BACKWARD_REDUCE_GLOBAL_TIMER = 'bwd_allreduce'? Can i choose one of them when i use Flops Profiler? If can, how?

BhAem avatar Aug 31 '24 05:08 BhAem