LouisSerrano

@lucidrains Ok great, thanks for the tips.

This was with my previous config. I benchmarked against fsq. I also show the aux_loss, which is going crazy for lfq_tanh

I think my main concern was on the auxiliary loss, I did not look to much into details but I assumed that somehow the model struggled to have a diverse...

@lucidrains I did not use VQ yet but I am gonna launch it tonight. So I'll let you know when I get the results. Also, I tested two recommended configurations...

@lucidrains I will give a shot to your config, thanks again !