A problem about the weight λ of Lvlb

[yinguanchun]

In the paper, λ is 0.001. The code sets learn_sigma as True and rescale_learned_sigmas as False, so the loss type will be gd.LossType.MSE, in this loss type ,the Lvlb will not multply 0.001. Even if the loss type is gd.LossType.RESCALED_MSE, terms["vb"] *= self.num_timesteps / 1000.0, what is self.num_timesteps, and what is its effect？ Thank you .

[zen-d]

@yinguanchun I am also confused about this scaling factor, have you understood that?

[Feynman1999]

I am also confused about this scaling factor, have you understood that?

[yhy258]

In my opinion, authors define L_{vlb} = L_0 + ... + L_T, not L_t. Thus, they may calculate the vlb loss with scale factor T (self.num_timestep).