xuwenju123

[BUG] DeadLock when using GQA and Mcore

2

**Describe the bug** I run the NeMo code and get the job stuck with pipeline_model_parallel_size > 1 using 8 GPUs. I run the job using the mainline NeMo and Megatron-lm...