Megatron-LM icon indicating copy to clipboard operation
Megatron-LM copied to clipboard

Published 20 hours ago • verified publisher icon NVIDIA

[QUESTION]Why forward_backward_pipelining_without_interleaving cannot open config.overlap_p2p_comm?

Open zhouyiyuan-mt opened this issue 1 year ago • 2 comments

forward_backward_pipelining_with_interleaving has a branch of opening config.overlap_p2p_comm, why does forward_backward_pipelining_without_interleaving not have?

zhouyiyuan-mt avatar Feb 04 '24 09:02 zhouyiyuan-mt

It can, we just haven't had the engineering resources to get around to it.

deepakn94 avatar Feb 08 '24 21:02 deepakn94

Marking as stale. No activity in 60 days.

github-actions[bot] avatar Apr 09 '24 18:04 github-actions[bot]