Random test_amberparams failure during Azure Pipeline

[lohedges]

I'm seeing this failure occasionally during the Azure Pipeline on macOS.

======================================================================
FAIL: test_amberparams.test_params
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/vsts/sire.app/lib/python3.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Users/vsts/sire.app/share/Sire/test/SireUnitTests/unittests/SireIO/test_amberparams.py", line 123, in test_params
    _assert_equal(oldparams, newparams)
  File "/Users/vsts/sire.app/share/Sire/test/SireUnitTests/unittests/SireIO/test_amberparams.py", line 103, in _assert_equal
    newparams.dihedralFunctions().potentials() )
  File "/Users/vsts/sire.app/share/Sire/test/SireUnitTests/unittests/SireIO/test_amberparams.py", line 65, in _assert_almost_equal
    assert_equal(op.atom2(), np.atom2())
AssertionError: {CGIdx(0),Index(20)} != {CGIdx(0),Index(23)}

It hasn't yet been triggered during the Linux build and I've failed to reproduce it locally.

[lohedges]

I've temporarily disabled the problematic test. It looks like other tests in this file have been commented out for similar reasons.

[chryswoods]

Thanks - this is a weird failure that does happen randomly only on OS X. It doesn’t affect the energy and, I think, is a result of the dihedrals sorting differently. It doesn’t affect energies or simulations so was not worth the effort to debug and remove. I left that test in as a reminder that it is an error. Maybe it could be marked as “expected” or “known” failure so that it doesn’t confuse people?

[lohedges]

Yes, that's a good idea, I'll look into expected failures with nose. I guess I might have to write a custom decorator since it only fails some of the time, i.e. we don't expect it to always fail. I read through the rest of that test set and came to the same conclusion regarding the dihedral re-ordering.

[lohedges]

Something like this, which is adatped from here:

import functools
import nose

def random_failure(test):
    @functools.wraps(test)
    def inner(*args, **kwargs):
        try:
            test(*args, **kwargs)
        except Exception:
            raise nose.SkipTest
    return inner