andersbll
Hyperparameters
Hello,
I have some questions regarding hyperparameters. I have made my own implementation in tensorflow. My question is what are your exact hyperparameter specifications for getting good results?
Currently I am using: lr (for all) --> 0.0003 l2_reg --> 0.0 batch_size --> 64 dropout --> keep probability = 1.0 Batch normalization RMSProp for the VAE modules and the discriminator. Not adapting learning rates. resize convolutions --> should I use transpose 2d Convolutions instead latent dimension --> 2048 60000 train images (CelebA) Not using full 200k at this point. encoder 64, 128, 256 all relu and batchnorm, + fully connected for sigma sq and mean z, no batchnorm after fully connected decoder fully connected (88256) + reshape + batchnorm + relu , conv layers 256, 128, 64, with batch norm and relu. fully connected as final layer with tanh.
I normalize the input images before the encoder to [-1, 1] and renormalize them after the tanh in the decoder to [0,1]
Discriminator conv layers: 32, 128, 256, 256, with batch norm and relu followed by a fully connected (512) relu layer with batch norm and finally a fully connected with dimension 1 and a sigmoid.
I have let this run for about 55 epochs and my reconstructed images look like this:
Does this setup seem right to you? When did you start getting more life like results?
kind regards,
Riaan
