Cheng Wan

Results 12 comments of Cheng Wan

[BugFix] Illegal memory access for MoE On H20

#3679 made the same fix 😂

Support qwen3 deepep

> > Do we need raise error for bf16 when enable deepep? > > I'm not sure, it's necessary? @zhyncs @ch-wan @sleepcoo How about raising NotImplementedError if it is not...

Extract generation_manager from tokenizer_manager

Thanks for the elaboration and efforts in addressing my questions. This PR LGTM now.

[Bug] when dp<ep, deepep moe occurs an error when running on 4*H800 nodes

Could you please check if #5657 resolves your issue? Here is the command I used to verify this PR, which works well: ``` # Node 1: python3 -m sglang.launch_server --model-path...

[Bug] when dp<ep, deepep moe occurs an error when running on 4*H800 nodes

Could you please share the command you used?

feat: mtp support dp-attention

@u4lr451 Could you please check if reverting #5657 helps? I'm going to fix the issue this weekend.

feat: mtp support dp-attention

@u4lr451 I have launched the test multiple times. The efficiency in our unit test for DeepSeek is consistently lower that the threshold (71 < 75). Could you please check if...

[bugfix] fix prefill TBO crashes when attn_tp_size > 1

Could add TBO to the deepep tests (Test 10-19 and Test 40-49) in https://github.com/sgl-project/sglang/blob/main/test/manual/ep/test_hybrid_dp_ep_tp_mtp.py to verify the functionality of your fix?

[bugfix] fix prefill TBO crashes when attn_tp_size > 1

/tag-and-rerun-ci

[bugfix] fix prefill TBO crashes when attn_tp_size > 1

@yuhyao Thank you for your excellent contribution. We do not plan to support TBO for allreduce / allgather-based dispatching.