megatron-lm topic

Annotations of the interesting ML papers I read

shreyansh26

Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*

xrsrke

A LLaMA1/LLaMA12 Megatron implement.

MoFHeka