Cheng Wan comments

Repositories
Issues
Comments

Results 12 comments of


                                            Cheng Wan

[Bug] fused_moe OOM when run deepseek-r1 with --speculative-algo NEXTN

The first cache `intermediate_cache1` can be released when we compute `intermediate_cache3`. How about reusing `intermediate_cache1`? We can achieve this by updating their initialization.

[Bug] sglang crashed when use enable_dp_attention running DeepSeekV3 on 2x8xH100

https://github.com/sgl-project/sglang/issues/4673 encounters the same issue. Follow https://github.com/sgl-project/sglang/issues/4673#issuecomment-2745578452 may fix your issue.