Convergence diagnostics

[dustinvtran]

Ideas

• Run chi-squared test sequentially after a batch of iterations to check convergence. This can also be used as a way to stop SGD early rather than running all of the iterations that a user may specify.
• Bootstrap data set, or choose different initial points, to see if the point estimate converges to something else
• Run a bunch of SGD chains, and analyze chain convergence using something like Gelman-Rubin diagnostic
• Assess (an estimated) number of remaining iterations for convergence while running sgd. This is similar to number of effective draws for MCMC.

[lantian2012]

Current implementation returns convergence=True without any checking