Error in the denoising score matching loss?

[lihenryhfl]

I may be wrong, but it appears to me that there is an error in the denoising score matching loss:

https://github.com/yang-song/score_sde_pytorch/blob/1618ddea340f3e4a2ed7852a0694a809775cf8d0/losses.py#L95

In particular, std, the denominator for z, should be squared, shouldn't it? This is what is prescribed in Eq. 5 in the earlier work [1]. Does this seem to be the case?

[1] Y. Song, S. Ermon, Generative Modeling by Estimating Gradients of the Data Distribution, 2019.

[yuanzhi-zhu]

Hey, you should notice that $\tilde{x}=x+\sigma z$ in Eq. 5, where $z$ is the standard gaussian distribution (hence it should be scaled by $\sigma$)