pydicom Test failures on aarch64

Describe the bug

=================================== FAILURES ===================================
_ TestPillowHandler_JPEG2K.test_array[/nix/store/jkrxsdz9l629y74n2jfbn9zf6g8j95c9-source/data_store/data/US1_J2KI.dcm-/nix/store/jkrxsdz9l629y74n2jfbn9zf6g8j95c9-source/data_store/data/US1_UNCI.dcm-fixes8] _

self = <pydicom.tests.test_pillow_pixel_data.TestPillowHandler_JPEG2K object at 0xffff97488ca0>
fpath = '/nix/store/jkrxsdz9l629y74n2jfbn9zf6g8j95c9-source/data_store/data/US1_J2KI.dcm'
rpath = '/nix/store/jkrxsdz9l629y74n2jfbn9zf6g8j95c9-source/data_store/data/US1_UNCI.dcm'
fixes = {}

    @pytest.mark.parametrize('fpath, rpath, fixes', JPEG2K_MATCHING_DATASETS)
    def test_array(self, fpath, rpath, fixes):
        """Test pixel_array returns correct values."""
        ds = dcmread(fpath)
        # May need to correct some element values
        for kw, val in fixes.items():
            setattr(ds, kw, val)
        arr = ds.pixel_array
        if 'YBR_FULL' in ds.PhotometricInterpretation:
            arr = convert_color_space(arr, ds.PhotometricInterpretation, 'RGB')
    
        ref = dcmread(rpath).pixel_array
>       assert np.array_equal(arr, ref)
E       assert False
E        +  where False = <function array_equal at 0xfffff69838b0>(array([[[0, 0, 0],\n        [0, 0, 0],\n        [0, 0, 0],\n        ...,\n        [0, 0, 0],\n        [0, 0, 0],\n        [0...     [0, 0, 0],\n        [0, 0, 0],\n        ...,\n        [1, 1, 1],\n        [1, 1, 1],\n        [1, 1, 1]]], dtype=uint8), array([[[0, 0, 0],\n        [0, 0, 0],\n        [0, 0, 0],\n        ...,\n        [0, 0, 0],\n        [0, 0, 0],\n        [0...     [0, 0, 0],\n        [0, 0, 0],\n        ...,\n        [1, 1, 1],\n        [1, 1, 1],\n        [1, 1, 1]]], dtype=uint8))
E        +    where <function array_equal at 0xfffff69838b0> = np.array_equal

pydicom/tests/test_pillow_pixel_data.py:443: AssertionError
_ TestPillowHandler_JPEG2K.test_array[/nix/store/jkrxsdz9l629y74n2jfbn9zf6g8j95c9-source/data_store/data/RG3_J2KI.dcm-/nix/store/jkrxsdz9l629y74n2jfbn9zf6g8j95c9-source/data_store/data/RG3_UNCI.dcm-fixes9] _

self = <pydicom.tests.test_pillow_pixel_data.TestPillowHandler_JPEG2K object at 0xffff9743dd30>
fpath = '/nix/store/jkrxsdz9l629y74n2jfbn9zf6g8j95c9-source/data_store/data/RG3_J2KI.dcm'
rpath = '/nix/store/jkrxsdz9l629y74n2jfbn9zf6g8j95c9-source/data_store/data/RG3_UNCI.dcm'
fixes = {}

    @pytest.mark.parametrize('fpath, rpath, fixes', JPEG2K_MATCHING_DATASETS)
    def test_array(self, fpath, rpath, fixes):
        """Test pixel_array returns correct values."""
        ds = dcmread(fpath)
        # May need to correct some element values
        for kw, val in fixes.items():
            setattr(ds, kw, val)
        arr = ds.pixel_array
        if 'YBR_FULL' in ds.PhotometricInterpretation:
            arr = convert_color_space(arr, ds.PhotometricInterpretation, 'RGB')
    
        ref = dcmread(rpath).pixel_array
>       assert np.array_equal(arr, ref)
E       assert False
E        +  where False = <function array_equal at 0xfffff69838b0>(array([[0, 0, 0, ..., 0, 0, 0],\n       [0, 0, 0, ..., 0, 0, 0],\n       [0, 0, 0, ..., 0, 0, 0],\n       ...,\n       [0, 0, 0, ..., 0, 0, 0],\n       [0, 0, 0, ..., 0, 0, 0],\n       [0, 0, 0, ..., 0, 0, 0]], dtype=uint16), array([[0, 0, 0, ..., 0, 0, 0],\n       [0, 0, 0, ..., 0, 0, 0],\n       [0, 0, 0, ..., 0, 0, 0],\n       ...,\n       [0, 0, 0, ..., 0, 0, 0],\n       [0, 0, 0, ..., 0, 0, 0],\n       [0, 0, 0, ..., 0, 0, 0]], dtype=uint16))
E        +    where <function array_equal at 0xfffff69838b0> = np.array_equal

pydicom/tests/test_pillow_pixel_data.py:443: AssertionError
_ TestPillowHandler_JPEG2K.test_array[/nix/store/jkrxsdz9l629y74n2jfbn9zf6g8j95c9-source/data_store/data/MR2_J2KI.dcm-/nix/store/jkrxsdz9l629y74n2jfbn9zf6g8j95c9-source/data_store/data/MR2_UNCI.dcm-fixes10] _

self = <pydicom.tests.test_pillow_pixel_data.TestPillowHandler_JPEG2K object at 0xffff96aeba90>
fpath = '/nix/store/jkrxsdz9l629y74n2jfbn9zf6g8j95c9-source/data_store/data/MR2_J2KI.dcm'
rpath = '/nix/store/jkrxsdz9l629y74n2jfbn9zf6g8j95c9-source/data_store/data/MR2_UNCI.dcm'
fixes = {}

    @pytest.mark.parametrize('fpath, rpath, fixes', JPEG2K_MATCHING_DATASETS)
    def test_array(self, fpath, rpath, fixes):
        """Test pixel_array returns correct values."""
        ds = dcmread(fpath)
        # May need to correct some element values
        for kw, val in fixes.items():
            setattr(ds, kw, val)
        arr = ds.pixel_array
        if 'YBR_FULL' in ds.PhotometricInterpretation:
            arr = convert_color_space(arr, ds.PhotometricInterpretation, 'RGB')
    
        ref = dcmread(rpath).pixel_array
>       assert np.array_equal(arr, ref)
E       assert False
E        +  where False = <function array_equal at 0xfffff69838b0>(array([[0, 0, 0, ..., 0, 0, 0],\n       [0, 0, 0, ..., 0, 0, 0],\n       [0, 0, 0, ..., 0, 0, 0],\n       ...,\n       [0, 0, 0, ..., 0, 0, 0],\n       [0, 0, 0, ..., 0, 0, 0],\n       [0, 0, 0, ..., 0, 0, 0]], dtype=uint16), array([[0, 0, 0, ..., 0, 0, 0],\n       [0, 0, 0, ..., 0, 0, 0],\n       [0, 0, 0, ..., 0, 0, 0],\n       ...,\n       [0, 0, 0, ..., 0, 0, 0],\n       [0, 0, 0, ..., 0, 0, 0],\n       [0, 0, 0, ..., 0, 0, 0]], dtype=uint16))
E        +    where <function array_equal at 0xfffff69838b0> = np.array_equal

pydicom/tests/test_pillow_pixel_data.py:443: AssertionError
_ TestPillowHandler_JPEG2K.test_array[/nix/store/jkrxsdz9l629y74n2jfbn9zf6g8j95c9-source/data_store/data/RG1_J2KI.dcm-/nix/store/jkrxsdz9l629y74n2jfbn9zf6g8j95c9-source/data_store/data/RG1_UNCI.dcm-fixes11] _

self = <pydicom.tests.test_pillow_pixel_data.TestPillowHandler_JPEG2K object at 0xffff96ba6be0>
fpath = '/nix/store/jkrxsdz9l629y74n2jfbn9zf6g8j95c9-source/data_store/data/RG1_J2KI.dcm'
rpath = '/nix/store/jkrxsdz9l629y74n2jfbn9zf6g8j95c9-source/data_store/data/RG1_UNCI.dcm'
fixes = {}

    @pytest.mark.parametrize('fpath, rpath, fixes', JPEG2K_MATCHING_DATASETS)
    def test_array(self, fpath, rpath, fixes):
        """Test pixel_array returns correct values."""
        ds = dcmread(fpath)
        # May need to correct some element values
        for kw, val in fixes.items():
            setattr(ds, kw, val)
        arr = ds.pixel_array
        if 'YBR_FULL' in ds.PhotometricInterpretation:
            arr = convert_color_space(arr, ds.PhotometricInterpretation, 'RGB')
    
        ref = dcmread(rpath).pixel_array
>       assert np.array_equal(arr, ref)
E       assert False
E        +  where False = <function array_equal at 0xfffff69838b0>(array([[18876, 18518, 18810, ...,  3920,  3679,  2279],\n       [18875, 18517, 18810, ...,  3920,  3680,  2279],\n      ...[20676, 20615, 20510, ...,  2445,  2639,  2962],\n       [20640, 20617, 20540, ...,  2423,  2723,  2856]], dtype=uint16), array([[18876, 18518, 18810, ...,  3920,  3679,  2279],\n       [18875, 18517, 18810, ...,  3920,  3680,  2279],\n      ...[20676, 20615, 20510, ...,  2445,  2639,  2962],\n       [20640, 20617, 20540, ...,  2423,  2723,  2856]], dtype=uint16))
E        +    where <function array_equal at 0xfffff69838b0> = np.array_equal

pydicom/tests/test_pillow_pixel_data.py:443: AssertionError

Steps To Reproduce Run pytest on aarch64.

Your environment

module       | version
------       | -------
platform     | Linux-5.11.14-aarch64-with-glibc2.17
Python       | 3.8.9 (default, Apr  2 2021, 11:20:07)  [GCC 9.3.0]
pydicom      | 2.1.2
gdcm         | _module not found_
jpeg_ls      | _module not found_
numpy        | 1.20.2
PIL          | 8.2.0

May 14 '21 09:05 dotlambda

Could you find out what version of openjpeg you're running?

May 14 '21 10:05 scaramallion

2.4.0

May 14 '21 10:05 dotlambda

Can't reproduce with Python 3.9, x86 Ubuntu 20.04, openjpeg 2.4, pillow 8.2. Hmm...

Could you run the following and attach the output dataset?

from pydicom import dcmread
from pydicom.data import get_testdata_file
from pydicom.uid import ExplicitVRLittleEndian

ds = dcmread(get_testdata_file("RG1_J2KI.dcm"))
ds.PixelData = ds.pixel_array.tobytes()
ds.file_meta.TransferSyntaxUID = ExplicitVRLittleEndian
ds.save_as("1386.dcm")

May 14 '21 11:05 scaramallion

And could you run opj_decompress -i rg1.jpg -o rg1.raw -OutFor RAW on the file below and attach it please?

Although its pretty large, you may want to zip it up first.

May 14 '21 12:05 scaramallion

Can't reproduce with Python 3.9, x86 Ubuntu 20.04, openjpeg 2.4, pillow 8.2. Hmm...

I can't reproduce it on x86_64-linux either, only on aarch64-linux.

I'll do the things you suggested, but it might take me a while, sorry.

May 14 '21 12:05 dotlambda

@dotlambda - is this still an issue? It's been a while...

Apr 19 '22 18:04 mrbean-bremen

Closing as unable to reproduce - reopen if necessary.

Apr 21 '23 19:04 darcymason