Saltychtao

I ran into the same problem when running on NVIDIA A100. I am using faiss-1.7.1 installed by pip.

I found when using EMA for cosine code book, the l2-norm of the input to the vq module would grow gradually, from 22 -> 20000, leading to growing training loss....

> I found when using EMA for cosine code book, the l2-norm of the input to the vq module would grow gradually, from 22 -> 20000, leading to growing training...

> @Saltychtao I also encounter a similar issue. Does vq_in refer to VectorQuantize.project_in? Yes.