Megatron-LM MuonClip support (non-split version)

MuonClip support (non-split version)

Open BoxiangW opened this issue 1 month ago • 11 comments

Added MLA and MHA(GQA) clipping support

Oct 24 '25 22:10 BoxiangW

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Oct 24 '25 22:10 copy-pr-bot[bot]

TE's https://github.com/NVIDIA/TransformerEngine/pull/2195 (2.9.0) is needed for this PR

Oct 24 '25 22:10 BoxiangW

TE's NVIDIA/TransformerEngine#2195 (2.9.0) is needed for this PR

It has been merged.

Oct 28 '25 23:10 skyw

/ok to test 7917e68

Nov 03 '25 06:11 BoxiangW

/ok to test 495f58d

Nov 03 '25 21:11 BoxiangW

/ok to test 55cc00d

Nov 04 '25 16:11 BoxiangW

/ok to test 9095615

Nov 05 '25 00:11 BoxiangW

Boxiang, I suppose you need someone in the expert reviewer list to review.

Nov 07 '25 16:11 skyw

/ok to test 1bb0407

Nov 10 '25 19:11 BoxiangW

/ok to test b63c573

Nov 10 '25 19:11 BoxiangW

/ok to test 95fdba3

Nov 20 '25 09:11 BoxiangW

Can we re-name this PR? It should just be "QK logits clipping" or something similar?

Nov 30 '25 17:11 deepakn94

/ok to test 6562a52

Dec 03 '25 18:12 BoxiangW