Megatron-LM icon indicating copy to clipboard operation
Megatron-LM copied to clipboard

Published 20 hours ago • verified publisher icon NVIDIA

Why is gather_output not supported in ColumnParallelLinear when using sequence parallelism?

Open mushan09 opened this issue 6 months ago • 1 comments

https://github.com/NVIDIA/Megatron-LM/blob/6bf8448ba065a0a37b2b874f49fd65ca9547b5c0/megatron/core/tensor_parallel/layers.py#L907

mushan09 avatar Aug 13 '24 06:08 mushan09