SGLD and pSGLD: incorrect weighting of noise term

[evolbio]

The noise term may be weighted incorrectly for SGLD and pSGLD. I am just learning this topic and apologize if I have misread the theory.

For SGLD, on page https://diffeqflux.sciml.ai/stable/examples/BayesianNODE_SGLD/, in the sgld() function, the variance of the noise should be , which means that randn in the noise term should be weighted by .

For preconditioned SGLD (pSGLD), the sample code is in https://github.com/RajDandekar/MSML21_BayesianNODE, in the files BayesiaNUDE_PSGLD_LV.jl and BayesianUDE_SEIR.jl. In that code, the preconditioning values are calculated as m. Let g = 1 ./ m, then the gradient is correctly weighted by g. However, the noise should be weighted by , which is not done.

[ChrisRackauckas]

@RajDandekar

[RajDandekar]

@evolbio I will take a look at this..

[ChrisRackauckas]

Removed.