kssmmm

Are there any updates regarding w2v-bert?

> Hi, Your training loss seems too low, should be ~1.4 after training for 200k steps and ~1.1 after 400k steps. super low loss in self-distillation usually means the teacher...