Siddharth Singh

icon company @NVIDIA icon location Santa Clara, CA icon location Research Scientist @ Nvidia, Building Megatron-inference.

Results 1 comments of Siddharth Singh

feat(MoE): Refactor cuda_graph_scope

If one wants to use per-layer cuda-graphs (--cuda-graph-scope full as of today in main), do we set --cuda-graph-scope as `attn mlp`? In that case, are we doubling the number of...