hahahaahaa

Tutel as an MoE backend in Nanotron for Qwen3-MoE 15B (128 experts, top-k=8)

3

Hello :) I’d like to use Tutel as the MoE layer implementation in Nanotron to train a **Qwen3-MoE** 15B model from scratch with 128 experts and top-k = 8. Cluster...