HodBadichi

[QUESTION] Measuring Pipeline Bubble Time During Megatron-LM Training

1

I'm curious about how you measured the precise bubble time during a run in your experiments(T_Comm in the paper). Megatron-LM's scheduling combines communication and idle time within the same NCCL...