sglang icon indicating copy to clipboard operation
sglang copied to clipboard

[CPU] Support chunk_gated_delta_rule kernel for Qwen3-Next

Open Valentine233 opened this issue 1 month ago • 8 comments

Motivation

This PR adds chunk_gated_delta_rule kernel for Qwen3-next.

Test Plan: test/srt/cpu/test_mamba.py -k test_chunk_gated_delta_rule

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

Valentine233 avatar Oct 31 '25 06:10 Valentine233

[!WARNING] You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

gemini-code-assist[bot] avatar Oct 31 '25 06:10 gemini-code-assist[bot]

@Valentine233 how much does this kernel contribute in e2e benchmarks right now?

mingfeima avatar Nov 07 '25 08:11 mingfeima

@Valentine233 how much does this kernel contribute in e2e benchmarks right now?

This kernel is about 13.67% of e2e, for Qwen3-Next prefill phase with BS=1, 1k length, TP=2 on GNR.

Valentine233 avatar Nov 10 '25 03:11 Valentine233

@Valentine233 need to update https://github.com/sgl-project/sglang/blob/main/test/srt/run_suite.py#L493-L510 to make CI really launch the test.

mingfeima avatar Nov 11 '25 06:11 mingfeima

@Valentine233 update this check util according this https://github.com/sgl-project/sglang/pull/12324#discussion_r2516428644

mingfeima avatar Nov 12 '25 02:11 mingfeima

fix CI fails.

mingfeima avatar Nov 12 '25 06:11 mingfeima

fix CI fails.

@mingfeima The CI failures seem not related with this PR. I rebase several times, but the failures still exist.

Valentine233 avatar Nov 12 '25 06:11 Valentine233

@Valentine233 Hi, could you plz fix lint? I will help you merge this PR.

FlamingoPg avatar Nov 22 '25 03:11 FlamingoPg

Thanks @FlamingoPg, the previous lint issue has been fixed. The current lint issue is not related to the PR: test/srt/test_priority_scheduling.py.

Valentine233 avatar Nov 24 '25 03:11 Valentine233

Hi @FlamingoPg, I have rebased again. There is no related CI issue now.

Valentine233 avatar Nov 26 '25 08:11 Valentine233