MOE with JetStream

[patrick-toulme]

Could someone explain or point to a doc that explains how MOE is implemented on Jetstream? Specifically, the all-to-all communications, static vs dynamic, sparse matmuls.

I would like to understand how XLA compiles MOE.