coniii icon indicating copy to clipboard operation
coniii copied to clipboard

RegularizedMeanField sometimes fails

Open artemmaksov opened this issue 6 years ago • 5 comments

I am trying to run RMF solver with synthetic data, but sometimes it results in an error. Here is the traceback:

File "python3.7/site-packages/coniii/solvers.py", line 1964, in solve solution = minimize_scalar(func) File "python3.7/site-packages/scipy/optimize/_minimize.py", line 770, in minimize_scalar return _minimize_scalar_brent(fun, bracket, args, **options) File "python3.7/site-packages/scipy/optimize/optimize.py", line 2141, in _minimize_scalar_brent brent.optimize() File "python3.7/site-packages/scipy/optimize/optimize.py", line 1925, in optimize xa, xb, xc, fa, fb, fc, funcalls = self.get_bracket_info() File "python3.7/site-packages/scipy/optimize/optimize.py", line 1899, in get_bracket_info xa, xb, xc, fa, fb, fc, funcalls = bracket(func, args=args) File "python3.7/site-packages/scipy/optimize/optimize.py", line 2324, in bracket fa = func(*(xa,) + args) File "python3.7/site-packages/coniii/solvers.py", line 1935, in func isingSamples = samples(J) File "python3.7/site-packages/coniii/solvers.py", line 1917, in samples self.multipliers = np.concatenate([J.diagonal(), squareform(mean_field_ising.zeroDiag(-J))]) File "python3.7/site-packages/scipy/spatial/distance.py", line 2193, in squareform is_valid_dm(X, throw=True, name='X') File "python3.7/site-packages/scipy/spatial/distance.py", line 2269, in is_valid_dm 'symmetric.') % name) ValueError: Distance matrix 'X' must be symmetric.

artemmaksov avatar Oct 17 '19 15:10 artemmaksov

Thanks for pointing this out.

My first guess is that there is some numerical precision error because the matrix J is symmetrized in the preceding line. Do you have a sense for what range of values your coupling matrix will be? For example, do you have nearly perfectly correlated spins?

eltrompetero avatar Oct 17 '19 15:10 eltrompetero

Thanks for the quick response! That's something I haven't really looked into. I have been using randomly generated matrix following an example in one of the notebooks and just changing value of n: h,J = np.random.normal(scale=.1,size=n),np.random.normal(scale=.1,size=n*(n-1)//2)

artemmaksov avatar Oct 17 '19 17:10 artemmaksov

I'm guessing that this problem occurs more frequently at smaller n. If that's the case, you may want to regularize the problem by disallowing correlations that are close to -1 or 1. That's where the parameters diverge.

Probably we should include a warning for such situations.

eltrompetero avatar Oct 17 '19 17:10 eltrompetero

Yes, I'm guessing that there's an issue with loss of precision or probably nans. Note squareform gives the same error for symmetric nans:

In [20]: scipy.spatial.distance.squareform([[0,1],[1,0]])
Out[20]: array([1])

In [21]: scipy.spatial.distance.squareform([[0,scipy.nan],[scipy.nan,0]])
ValueError: Distance matrix 'X' must be symmetric.

Maybe we need to check for nans in the J matrix?

bcdaniels avatar Nov 25 '19 11:11 bcdaniels

@bcdaniels The RMF code is a bit opaque to me right now. Could you point out where nans could appear?

eltrompetero avatar Nov 28 '19 07:11 eltrompetero