bug fix IWAE: use SGVB1 instead of SGVB2, detach sample weights gradi…

[tongdaxu]

Hi thanks for this amazing project, I like it.

I find an implementation detail of Importance Weighted Autoencoder concerning:

• First, it is SGVB 1 (Auto-encoding Variational Bayes Eq 6) should be used instead of SGVB 2 (Auto-encoding Variational Bayes Eq 7, 10) for estimating the evidence lower bound
• Second, the importance weights should be detached, or the gradient is different from (Importance Weighted Autoencoder Eq 14). To be specific:
  • in EQ 14: grad = \Sum w * \nabla ELBO
  • in current impl, grad = \Sum \nabla (w * ELBO) = \Sum w * \nabla ELBO + \nabla w * EBLO, an extra term is introduced

I made a loosy PR as I do not know how to run this repo properly. I would appreciate any effort to review and correct it.

Thanks

[tongdaxu]

This problem is elaborated in issue: https://github.com/AntixK/PyTorch-VAE/issues/34.