lightly
lightly copied to clipboard
lightly-ai
Support Numpy 2.0
Error using lightly with numpy 2.0
Numpy 2.0.0 was released on June 16th 2024 and does not work with lightly.
Using numpy==2.0.0 and lightly==1.5.6 and python==3.10.8:
(.venv) user@device repo % python -c "import lightly"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/malteebnerlightly/Documents/GitHub/lightspeed/.venv/lib/python3.10/site-packages/lightly/__init__.py", line 88, in <module>
from lightly.api import _version_checking
File "/Users/malteebnerlightly/Documents/GitHub/lightspeed/.venv/lib/python3.10/site-packages/lightly/api/__init__.py", line 7, in <module>
from lightly.api.api_workflow_client import ApiWorkflowClient
File "/Users/malteebnerlightly/Documents/GitHub/lightspeed/.venv/lib/python3.10/site-packages/lightly/api/api_workflow_client.py", line 21, in <module>
from lightly.api.api_workflow_selection import _SelectionMixin
File "/Users/malteebnerlightly/Documents/GitHub/lightspeed/.venv/lib/python3.10/site-packages/lightly/api/api_workflow_selection.py", line 30, in <module>
class _SelectionMixin:
File "/Users/malteebnerlightly/Documents/GitHub/lightspeed/.venv/lib/python3.10/site-packages/lightly/api/api_workflow_selection.py", line 32, in _SelectionMixin
self, al_scores: Dict[str, NDArray[np.float_]], query_tag_id: str
File "/Users/malteebnerlightly/Documents/GitHub/lightspeed/.venv/lib/python3.10/site-packages/numpy/__init__.py", line 397, in __getattr__
raise AttributeError(
AttributeError: `np.float_` was removed in the NumPy 2.0 release. Use `np.float64` instead.. Did you mean: 'float16'?
Reproduce
Run in a new venv, such that numpy 2.0 is installed:
pip install lightly
python -c "import lightly"
Workaround
Just install an older numpy version, e.g. using pip install numpy==1.26.
It looks like torchvision 0.19 will be compatible with numpy 2.0 (source: https://github.com/pytorch/vision/issues/8460#issuecomment-2145217340). Let's require numpy 1.x for now.
Why can't we support both numpy 1 and 2 as soon as possible? Why do we need to wait for other dependencies? Seems like it's a simple fix.
Hey @adamjstewart, thanks for the feedback. We can already migrate the library to NumPy 2.0 as in #1561 but the problem of broken upstream libraries remains. In particular, torchvision 0.18.0 is only 98% compatible (see the tracker issue https://github.com/numpy/numpy/issues/26191) and it appears that we are affected by the remaining two percent: Applying color jitter results in the following error (failing GitHub action):
transformed: Tensor = self.transform(image)
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/torchvision/transforms/transforms.py:95: in __call__
img = t(img)
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/torch/nn/modules/module.py:1532: in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/torch/nn/modules/module.py:1541: in _call_impl
return forward_call(*args, **kwargs)
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/torchvision/transforms/transforms.py:540: in forward
img = t(img)
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/torch/nn/modules/module.py:1532: in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/torch/nn/modules/module.py:1541: in _call_impl
return forward_call(*args, **kwargs)
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/torchvision/transforms/transforms.py:1280: in forward
img = F.adjust_hue(img, hue_factor)
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/torchvision/transforms/functional.py:959: in adjust_hue
return F_pil.adjust_hue(img, hue_factor)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
img = <PIL.Image.Image image mode=RGB size=100x100 at 0x7FAC861DADA0>
hue_factor = -0.020240534096956253
@torch.jit.unused
def adjust_hue(img: Image.Image, hue_factor: float) -> Image.Image:
if not (-0.5 <= hue_factor <= 0.5):
raise ValueError(f"hue_factor ({hue_factor}) is not in [-0.5, 0.5].")
if not _is_pil_image(img):
raise TypeError(f"img should be PIL Image. Got {type(img)}")
input_mode = img.mode
if input_mode in {"L", "1", "I", "F"}:
return img
h, s, v = img.convert("HSV").split()
np_h = np.array(h, dtype=np.uint8)
# uint8 addition take cares of rotation across boundaries
with np.errstate(over="ignore"):
> np_h += np.uint8(hue_factor * 255)
E OverflowError: Python integer -5 out of bounds for uint8
/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/torchvision/transforms/_functional_pil.py:114: OverflowError
It's not possible for your tests to pass without waiting for torchvision, but it is possible for the next release to be compatible with numpy 2 (futureproof). This way, as soon as torchvision makes a new release, the last month of lightly releases will already immediately support numpy 2.
We appreciate your input. However, relaxing the numpy version requirement again will lead to more broken installations, which we want to avoid. We've chosen to keep numpy constrained to <2.0 for the time being.
For the moment, there is no torchvision version compatible with numpy 2 thus we restricted the requirements to not use numpy 2: https://github.com/lightly-ai/lightly/pull/1562
Once a compatible torchvision version is released, we can have a look again.
No rush, lightly's release schedule is already faster than most. This is partly why I was so adamant about supporting numpy 2 immediately and letting torchvision deal with complaints about numpy 2 incompatibility
Unfortunately torch/torchvision still don't support numpy 2 on Windows in the latest release, but at least we can use this on macOS/Linux.
Conda-forge still needs to be updated with NumPy 2 support: https://github.com/conda-forge/lightly-feedstock/issues/20