sonnet
sonnet copied to clipboard
Can you explain the logic of updated_ema_cluster_size in VectorQuantizerEMA?
Hi,
It seems updated_ema_cluster_size
in VectorQuantizerEMA
use small epsilon to avoid zero in denominator, but the formula is strange to me, so can you explain the logic here, and why not directly use something like x=x+epsilon
?
https://github.com/deepmind/sonnet/blob/d1cd37117bcb98223b3e4b930717d418abb76484/sonnet/src/nets/vqvae.py#L270-L271