edward icon indicating copy to clipboard operation
edward copied to clipboard

Published 20 hours ago • verified publisher icon blei-lab

VAE: bad convergence of loss (attached tensorboard image)

Open pauloabelha opened this issue 7 years ago • 5 comments

HI, I've been diving deeper into VAE and using Edward as my experimental toolbox.

I ran the exact same code as examples/vae.py, only altering one line to get tensorboard data: Line 77: inference.initialize(optimizer=optimizer, logdir="/tmp/log")

When I check my tensorboard I see a very bad convergence of loss function (please see tensorboard image attached for a complete run of the code).

Final printed lower bound: -log p(x) <= 176.285 ed_vae_tensorboard

Am I missing something?

Thank you,

pauloabelha avatar Feb 16 '18 16:02 pauloabelha

Nope. You are correct. Happy to take a contribution changing the number of iterations to one where it stabilizes.

dustinvtran avatar Feb 16 '18 17:02 dustinvtran

OK. Will work on it.

pauloabelha avatar Feb 16 '18 18:02 pauloabelha

To those also thinking of working on this,

I had to stop the running after two days. I need a better computer. I got to: -log p(x) <= 207.419 Epoch: 17768

See attached image for tensorboard. ed_vae_tensorboard_20k

pauloabelha avatar Feb 19 '18 10:02 pauloabelha

It seems unnecessary to be super-specific about the number of training iterations if it takes that long. Maybe try larger batch sizes (e.g., 256) and a non-trivial latent dimension (e.g., 50)?

dustinvtran avatar Feb 20 '18 08:02 dustinvtran

Yes, thank you. I’ll experiment. I just had to stop for a bit and decided to dump this result here for anyone interested in the issue.

pauloabelha avatar Feb 20 '18 08:02 pauloabelha