Unit test failure on s389x

[avalentino]

Problem description

Some of the tests run on the Debian CI infrastructure are failing on s390x architectures [1, 2]. Actually only one test is failing check_backwards_compatibility probably due to some kind of incompatibility with BE architectures (related issue #187).

[1] https://buildd.debian.org/status/fetch.php?pkg=numcodecs&arch=s390x&ver=0.7.2%2Bds-1&stamp=1601282966&raw=0 [2] https://buildd.debian.org/status/package.php?p=numcodecs

============================= test session starts ==============================
platform linux -- Python 3.8.6, pytest-4.6.11, py-1.9.0, pluggy-0.13.0
rootdir: /<<PKGBUILDDIR>>, inifile: pytest.ini
collected 522 items

[CUT]


=================================== FAILURES ===================================
_________________________ test_backwards_compatibility _________________________

    def test_backwards_compatibility():
        codec = Categorize(labels=labels, dtype='<U4', astype='u1')
>       check_backwards_compatibility(Categorize.codec_id, arrays, [codec],
                                      prefix='U')

numcodecs/tests/test_categorize.py:80: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

codec_id = 'categorize'
arrays = [array(['ßàř', 'ßàř', 'ßāẑ', 'ƪùüx', 'ƒöõ', 'ƒöõ', 'ßàř', 'ßāẑ', 'ßāẑ',
       'ƒöõ', 'ßāẑ', 'ƒöõ', 'ßāẑ', 'ƪùüx', 'ƪù...        'ƒöõ'],
       ['ƒöõ', 'ßàř', 'ƒöõ', 'ƪùüx', 'ƒöõ', 'ƪùüx', 'ßàř', 'ßàř', 'ƒöõ',
        'ƒöõ']], dtype='>U4')]
codecs = [Categorize(dtype='<U4', astype='|u1', labels=['ƒöõ', 'ßàř', 'ßāẑ', ...])]
precision = None, prefix = 'U'

    def check_backwards_compatibility(codec_id, arrays, codecs, precision=None, prefix=None):
    
        # setup directory to hold data fixture
        if prefix:
            fixture_dir = os.path.join('fixture', codec_id, prefix)
        else:
            fixture_dir = os.path.join('fixture', codec_id)
        if not os.path.exists(fixture_dir):  # pragma: no cover
            os.makedirs(fixture_dir)
    
        # save fixture data
        for i, arr in enumerate(arrays):
            arr_fn = os.path.join(fixture_dir, 'array.{:02d}.npy'.format(i))
            if not os.path.exists(arr_fn):  # pragma: no cover
                np.save(arr_fn, arr)
    
        # load fixture data
        for arr_fn in glob(os.path.join(fixture_dir, 'array.*.npy')):
    
            # setup
            i = int(arr_fn.split('.')[-2])
            arr = np.load(arr_fn, allow_pickle=True)
            arr_bytes = arr.tobytes(order='A')
            if arr.flags.f_contiguous:
                order = 'F'
            else:
                order = 'C'
    
            for j, codec in enumerate(codecs):
    
                # setup a directory to hold encoded data
                codec_dir = os.path.join(fixture_dir, 'codec.{:02d}'.format(j))
                if not os.path.exists(codec_dir):  # pragma: no cover
                    os.makedirs(codec_dir)
    
                # file with codec configuration information
                codec_fn = os.path.join(codec_dir, 'config.json')
    
                # one time save config
                if not os.path.exists(codec_fn):  # pragma: no cover
                    with open(codec_fn, mode='w') as cf:
                        _json.dump(codec.get_config(), cf, sort_keys=True, indent=4)
    
                # load config and compare with expectation
                with open(codec_fn, mode='r') as cf:
                    config = _json.load(cf)
                    assert codec == get_codec(config)
    
                enc_fn = os.path.join(codec_dir, 'encoded.{:02d}.dat'.format(i))
    
                # one time encode and save array
                if not os.path.exists(enc_fn):  # pragma: no cover
                    enc = codec.encode(arr)
                    enc = ensure_bytes(enc)
                    with open(enc_fn, mode='wb') as ef:
                        ef.write(enc)
    
                # load and decode data
                with open(enc_fn, mode='rb') as ef:
                    enc = ef.read()
                    dec = codec.decode(enc)
                    dec_arr = ensure_ndarray(dec).reshape(-1, order='A')
                    dec_arr = dec_arr.view(dtype=arr.dtype).reshape(arr.shape, order=order)
                    if precision and precision[j] is not None:
                        assert_array_almost_equal(arr, dec_arr, decimal=precision[j])
                    elif arr.dtype == 'object':
                        assert_array_items_equal(arr, dec_arr)
                    else:
>                       assert_array_equal(arr, dec_arr)
E                       AssertionError: 
E                       Arrays are not equal
E                       
E                       Mismatched elements: 1000 / 1000 (100%)
E                        x: array([['ßāẑ', 'ßàř', 'ßàř', 'ßāẑ', 'ßāẑ', 'ßàř', 'ßāẑ', 'ƒöõ', 'ßāẑ',
E                               'ßāẑ'],
E                              ['ßàř', 'ƪùüx', 'ƪùüx', 'ßàř', 'ƪùüx', 'ßāẑ', 'ƪùüx', 'ƒöõ',...
E                        y: array([['', '', '', '', '', '', '', '', '', ''],
E                              ['', '', '', '', '', '', '', '', '', ''],
E                              ['', '', '', '', '', '', '', '', '', ''],...

numcodecs/tests/common.py:302: AssertionError
=============================== warnings summary ===============================
.pybuild/cpython3_3.8_numcodecs/build/numcodecs/tests/test_json.py::test_non_numpy_inputs
.pybuild/cpython3_3.8_numcodecs/build/numcodecs/tests/test_msgpacks.py::test_non_numpy_inputs
  /usr/lib/python3/dist-packages/numpy/core/_asarray.py:83: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
    return array(a, dtype, copy=False, order=order)

.pybuild/cpython3_3.8_numcodecs/build/numcodecs/tests/test_json.py::test_non_numpy_inputs
  /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_numcodecs/build/numcodecs/tests/test_json.py:73: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
    assert np.array_equal(np.array(input_data), output_data)

.pybuild/cpython3_3.8_numcodecs/build/numcodecs/tests/test_msgpacks.py::test_non_numpy_inputs
  /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_numcodecs/build/numcodecs/tests/test_msgpacks.py:76: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
    expect = np.array(input_data)

-- Docs: https://docs.pytest.org/en/latest/warnings.html
=== 1 failed, 481 passed, 1 skipped, 39 xfailed, 4 warnings in 11.03 seconds ===

Version and installation information

Please provide the following:

• Value of numcodecs.__version__: 0.7.2
• Version of Python interpreter: 3.8.6
• Operating system (Linux/Windows/Mac): GNU/Linux (Debian sid)
• How NumCodecs was installed (e.g., "using pip into virtual environment", or "using conda"): apt

[avalentino]

Any comment about this? Is it a real issue or is it maybe a small bug in the test code?

[QuLogic]

Odd, this doesn't fail on Fedora s390x. Maybe it has something to do with the random choice of arrays?

[antkneed]

I am not sure of the root cause, but I believe that Fedora is succeeding because they use the pre-built fixtures / config for the tests: see here.

Hopefully this helps narrow down what the best fix / workaround is. 🤔