simsopt icon indicating copy to clipboard operation
simsopt copied to clipboard

test_meansquaredforces_taylor_test failing on random runners

Open smiet opened this issue 1 year ago • 0 comments

Noticed that the test_meansquaredforces_taylor_test sporadically fails. These were added for the coil force PR #450, can you have a look @smhurwitz ?

See for example this test

Error message:

======================================================================
FAIL: test_meansquaredforces_taylor_test (field.test_selffieldforces.CoilForcesTest)
Verify that dJ matches finite differences of J
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/simsopt/simsopt/tests/field/test_selffieldforces.py", line 284, in test_meansquaredforces_taylor_test
    np.testing.assert_array_less(err_new, 0.3 * err)
  File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 1112, in assert_array_less
    assert_array_compare(operator.__lt__, x, y, err_msg=err_msg,
  File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 797, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not less-ordered

Mismatched elements: 1 / 1 (100%)
Max absolute difference: 0.34464977
Max relative difference: 32386.30997776
 x: array(0.34466)
 y: array(1.064184e-05)**

I don't understand why it would fail on two runners but be fine on all others. I could suggest using a simple assert instead of np.testing.assert_array_less for a clearer error trace.

smiet avatar Oct 21 '24 14:10 smiet