sonnet icon indicating copy to clipboard operation
sonnet copied to clipboard

Can you explain the logic of updated_ema_cluster_size in VectorQuantizerEMA?

Open wztdream opened this issue 2 years ago • 0 comments

Hi, It seems updated_ema_cluster_size in VectorQuantizerEMA use small epsilon to avoid zero in denominator, but the formula is strange to me, so can you explain the logic here, and why not directly use something like x=x+epsilon? https://github.com/deepmind/sonnet/blob/d1cd37117bcb98223b3e4b930717d418abb76484/sonnet/src/nets/vqvae.py#L270-L271

wztdream avatar Jul 21 '22 03:07 wztdream