Can DeepEP run correctly in cudaGraph mode?

[ghostplant]

Two questions:

1. Can DeepEP run normally in cudaGraph mode?
2. Does DeepEP perform "dropless" MoE dispatch? (i.e. no token discarded if tokens are heavily routed to a limited number of experts)

[LyricZhao]

1. Yes, but only for the normal kernels;
2. Yes; If you want to drop tokens, you should perform at the gate (masking some topk_idx into -1), DeepEP supports ignoring -1 expert selection (no send for such cases).

[ghostplant]

@LyricZhao Thank you, for drop-less dispatch, will the utilization still be that fast when gating selection is imbalanced (e.g. all tokens routed to the same GPU)?

[LyricZhao]

will the utilization still be that fast when gating selection is imbalanced

The overall performance will be bound at the imbalanced rank. In the terms of the imbalanced rank itself, the utilization should be full.

[alpha-baby]

1. Yes, but only for the normal kernels;

In sglang, low-latancy kernel could run normally in cudaGraph Mode!

Why only for the normal kernels? @LyricZhao