GQN Loss and results

[BingCS]

Hi,

Many thanks for your contribution on implementing GQN.

Recently, I implemented GQN. After the training on rooms_ring_camera, I found that the loss will converge to around 6950 after 200K steps, which means after the sigma decrease to 0.7, the loss will maintain at a stable level. I noticed that your loss also converge to 6950. But in their original paper, the final loss is just 6.1. I don’t understand the where is the problem. Many thanks in advance!

Looking forward to your reply!

[musyoku]

Hi. I think this figure in the original paper shows the value of bits/pixel.

bits/pixel is calculated as follows:

bits_per_pixel = -(-ELBO / num_pixels_per_batch - np.log(128)) / np.log(
            2)

https://github.com/musyoku/chainer-gqn/blob/master/train.py#L206

num_pixels_per_batch is 3 * 64 * 64 in this time, so bits/pixel will be 6.184 when ELBO is -6950.

See also: https://www.reddit.com/r/MachineLearning/comments/56m5o2/discussion_calculation_of_bitsdims/

[BingCS]

Great thanks!

On 29 Jul 2019, at 10:52, musyoku [email protected] wrote:

Hi. This figure in the original paper shows the value of bits/pixel. https://user-images.githubusercontent.com/15250418/62034651-5d495100-b229-11e9-8daa-1d7e72e6807f.png bits/pixel is calculated as follows:

bits_per_pixel = -(-ELBO / num_pixels_per_batch - np.log(128)) / np.log( 2) https://github.com/musyoku/chainer-gqn/blob/master/train.py#L206 https://github.com/musyoku/chainer-gqn/blob/master/train.py#L206 num_pixels_per_batch is 3 * 64 * 64 in this time, so bits/pixel will be 6.184 when ELBO is -6950.

See also: https://www.reddit.com/r/MachineLearning/comments/56m5o2/discussion_calculation_of_bitsdims/ https://www.reddit.com/r/MachineLearning/comments/56m5o2/discussion_calculation_of_bitsdims/ — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/musyoku/chainer-gqn/issues/17?email_source=notifications&email_token=AH2LQRAKUPO76DAQD7FK74LQB24WLA5CNFSM4IHK2RUKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3AGQWA#issuecomment-515926104, or mute the thread https://github.com/notifications/unsubscribe-auth/AH2LQRH4MGSQFUKVFMALRD3QB24WLANCNFSM4IHK2RUA.

[BingCS]

Hi,

During training, the loss is -ELBO(loss = -ELBO) and we want to minimise it. But for bits_per_pixel from the above equation, it will increase when loss decrease.

[musyoku]

Sorry I made mistakes. The correct formula is

bits_per_pixel = -(ELBO / num_pixels_per_batch - np.log(256)) / np.log(
            2)

In my code, the pixel value will be scaled to [-0.5, 0.5], so we need to subtract log(256) to account for this rescaling.

When ELBO is -6950, bits/pixel = -(-6950 / (3 * 64 * 64) - log(128)) / log(2) = 8.8159 This value is different from one reported in the paper, so something may still be wrong. :worried: