bioio Remove numpy<2.0.0 restriction

Ideally bioio should support the latest numpy.

I checked with the ruff rule for numpy 2.0 migration, which is not (cannot possibly be) 100% comprehensive and found no issues in bioio, bioio-base, and bioio-ome-zarr. Does anyone (@SeanLeRoy) know of anything that would be broken by numpy 2?

If necessary, individual bioio plugins could retain the numpy<2.0.0 restriction until they are updated for compatibility.

Aug 20 '24 23:08 pgarrison

Currently fails when unpinning numpy (upgraded to 2.1.1) (Python 3.10)

2024-09-25 09:41:09.904 [error] pytest test discovery error for workspace:  /Users/brian.whitney/Desktop/Repos/bioio 
 ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
 Check Python Test Logs for more details.

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
 Check Python Test Logs for more details.

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
 Check Python Test Logs for more details.

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
 Check Python Test Logs for more details.

Interrupted: 4 errors during collection
 Check Python Test Logs for more details.
2024-09-25 09:41:09.906 [error] pytest test discovery error for workspace:  /Users/brian.whitney/Desktop/Repos/bioio 
 ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
 Check Python Test Logs for more details.

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
 Check Python Test Logs for more details.

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
 Check Python Test Logs for more details.

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
 Check Python Test Logs for more details.

Interrupted: 4 errors during collection
 Check Python Test Logs for more details.
2024-09-25 09:41:09.906 [info] 
==================================== ERRORS ====================================
_________ ERROR collecting bioio/tests/writers/test_ome_tiff_writer.py _________
bioio/tests/writers/test_ome_tiff_writer.py:14: in <module>
    from bioio.writers import OmeTiffWriter
bioio/writers/__init__.py:5: in <module>
    from .ome_zarr_writer import OmeZarrWriter
bioio/writers/ome_zarr_writer.py:7: in <module>
    from ome_zarr.scale import Scaler
../../../.pyenv/versions/3.10.14/envs/celigo-pipeline-service/lib/python3.10/site-packages/ome_zarr/scale.py:17: in <module>
    from skimage.transform import (
../../../.pyenv/versions/3.10.14/envs/celigo-pipeline-service/lib/python3.10/site-packages/skimage/__init__.py:122: in <module>
    from ._shared import geometry
geometry.pyx:1: in init skimage._shared.geometry
    ???
E   ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
_________ ERROR collecting bioio/tests/writers/test_ome_zarr_writer.py _________
bioio/tests/writers/test_ome_zarr_writer.py:13: in <module>
    from bioio.writers import OmeZarrWriter
bioio/writers/__init__.py:5: in <module>
    from .ome_zarr_writer import OmeZarrWriter
bioio/writers/ome_zarr_writer.py:7: in <module>
    from ome_zarr.scale import Scaler
../../../.pyenv/versions/3.10.14/envs/celigo-pipeline-service/lib/python3.10/site-packages/ome_zarr/scale.py:17: in <module>
    from skimage.transform import (
../../../.pyenv/versions/3.10.14/envs/celigo-pipeline-service/lib/python3.10/site-packages/skimage/__init__.py:122: in <module>
    from ._shared import geometry
geometry.pyx:1: in init skimage._shared.geometry
    ???
E   ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
________ ERROR collecting bioio/tests/writers/test_timeseries_writer.py ________
bioio/tests/writers/test_timeseries_writer.py:12: in <module>
    from bioio.writers.timeseries_writer import TimeseriesWriter
bioio/writers/__init__.py:5: in <module>
    from .ome_zarr_writer import OmeZarrWriter
bioio/writers/ome_zarr_writer.py:7: in <module>
    from ome_zarr.scale import Scaler
../../../.pyenv/versions/3.10.14/envs/celigo-pipeline-service/lib/python3.10/site-packages/ome_zarr/scale.py:17: in <module>
    from skimage.transform import (
../../../.pyenv/versions/3.10.14/envs/celigo-pipeline-service/lib/python3.10/site-packages/skimage/__init__.py:122: in <module>
    from ._shared import geometry
geometry.pyx:1: in init skimage._shared.geometry
    ???
E   ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
__________ ERROR collecting bioio/tests/writers/test_two_d_writer.py ___________
bioio/tests/writers/test_two_d_writer.py:12: in <module>
    from bioio.writers.two_d_writer import TwoDWriter
bioio/writers/__init__.py:5: in <module>
    from .ome_zarr_writer import OmeZarrWriter
bioio/writers/ome_zarr_writer.py:7: in <module>
    from ome_zarr.scale import Scaler
../../../.pyenv/versions/3.10.14/envs/celigo-pipeline-service/lib/python3.10/site-packages/ome_zarr/scale.py:17: in <module>
    from skimage.transform import (
../../../.pyenv/versions/3.10.14/envs/celigo-pipeline-service/lib/python3.10/site-packages/skimage/__init__.py:122: in <module>
    from ._shared import geometry
geometry.pyx:1: in init skimage._shared.geometry
    ???
E   ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
=========================== short test summary info ============================
ERROR bioio/tests/writers/test_ome_tiff_writer.py - ValueError: numpy.dtype s...
ERROR bioio/tests/writers/test_ome_zarr_writer.py - ValueError: numpy.dtype s...
ERROR bioio/tests/writers/test_timeseries_writer.py - ValueError: numpy.dtype...
ERROR bioio/tests/writers/test_two_d_writer.py - ValueError: numpy.dtype size...
!!!!!!!!!!!!!!!!!!! Interrupted: 4 errors during collection !!!!!!!!!!!!!!!!!!!!
===================== 2 tests collected, 4 errors in 0.16s =====================

Sep 25 '24 16:09 BrianWhitneyAI

@BrianWhitneyAI That error message reminds me of errors that can come up when using pandas<2 with numpy>=2

Oct 08 '24 19:10 pgarrison

bioio-bioformats also is showing some numpy2 problems in the actions runs.

bioio_bioformats/__init__.py:15: in <module>
    from .reader import Reader
bioio_bioformats/reader.py:8: in <module>
    from bioio_base import constants, dimensions, exceptions, io, reader, types
/opt/hostedtoolcache/Python/3.9.[20](https://github.com/bioio-devs/bioio-bioformats/actions/runs/12125702446/job/33806230021#step:9:21)/x64/lib/python3.9/site-packages/bioio_base/io.py:12: in <module>
    from .types import PathLike
/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/bioio_base/types.py:7: in <module>
    import dask.array as da
/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/dask/array/__init__.py:2: in <module>
    from dask.array import backends, fft, lib, linalg, ma, overlap, random
/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/dask/array/ma.py:8: in <module>
    from dask.array.routines import _average
/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/dask/array/routines.py:1569: in <module>
    @implements(np.round, np.round_)
/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/numpy/__init__.py:397: in __getattr__
    raise AttributeError(
E   AttributeError: `np.round_` was removed in the NumPy 2.0 release. Use `np.round` instead.

maybe a conflict between dask and numpy versions...

Dec 03 '24 20:12 toloudis

ome-zarr-py requires requires-python = ">3.10" as of 0.10.3. (discovered while looking into this issue)

Jan 22 '25 17:01 kmitcham

also see #80 for discussion around increasing minimum python version required

Jan 30 '25 22:01 toloudis

Looks like the future makes the problem go away, if only we could reach the future: ====================================================================================================== 219 passed, 75 xfailed, 14119 warnings in 152.60s (0:02:32) ======================================================================================================= (venv-3.13.0) kevin.mitcham@ip-10-141-0-41:~/projects/bioio$ pip freeze | grep numpy numpy==2.2.2 numpydoc==1.8.0 (venv-3.13.0) kevin.mitcham@ip-10-141-0-41:~/projects/bioio$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 20.04.6 LTS Release: 20.04 Codename: focal just test passes on python 3.13, ubuntu 20.04

Feb 05 '25 20:02 kmitcham

Going to put this on the backburned until https://github.com/bioio-devs/bioio/pull/80 gets merged in

Feb 11 '25 23:02 kmitcham

I think the release that unblocks https://github.com/bioio-devs/bioio/pull/80 has been merged, so 80 might move soon.

Mar 20 '25 22:03 kmitcham