Confusion about the mutual information metric

[smolPixel]

Hello, thank you very much for making the code available. I'm confused about the mutual information math, more specifically about the line

E_{q(z|x)}log(q(z|x)) = -0.5nzlog(2*\pi) - 0.5*(1+logvar).sum(-1) neg_entropy = (-0.5 * nz * math.log(2 * math.pi) - 0.5 * (1 + logv).sum(-1)).mean()

When I derive it, it gives me neg_entropy = (-0.5 * nz * math.log(2 * math.pi) - 0.5 * (logv).sum(-1)).mean()

So I think I must have made a mistake somewhere? Thank you

[seongminp]

Hello. I followed the calculations provided in previous research.

(The link is not the original.. If you search for calc_mi, several VAE repos come up.)

[smolPixel]

Thank you for your fast answer, I will try to see from previous research. Would you have by any chance any reference which show the derivation? From what I can see everyone cite Hoffman and Johnson in their paper, which calculate MI between the code and the index by subtracting terms to the ELBO, but calculate in github the MI between X and Z by analytical solution+estimation of the aggregate posterior, following, I think https://github.com/jxhe/vae-lagging-encoder/blob/master/modules/encoders/encoder.py. I'm not sure my math is right or if the two ways are equivalent but I can't find any papers that show the derivation. Thank you very much.