Fix byte/int8 overflow with numpy >= 2.0
** I used to request merging into "dev" branch but it seems to be stale - please change if this is the incorrect branch
Before numpy 2.0 a value of 255 was automatically converted to -1 for int8/byte data type. Last numpy versions from 1.x series show "deprecation warning" but after 2.x it causes a overflow exception and application crashes.
This change converts data to a larger data size before converting it back to int8/byte. This should give same result.
I was not able to test it on s RMS/jessie image, but I tested the approach on python 2.7. See bellow the results
Python 2.7.18 (default, Apr 20 2020, 19:27:10)
[GCC 8.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> (np.array(np.array([[0, -128], [-100, 100], [37, -80], [255, 0]], np.byte))*255).astype(np.byte) # deprecation warning
array([[ 0, -128],
[ 100, -100],
[ -37, 80],
[ 1, 0]], dtype=int8)
>>>
>>>
>>> (np.array(np.array([[0, -128], [-100, 100], [37, -80], [255, 0]], np.int16)).astype(np.byte)*255).astype(np.byte) # ok
array([[ 0, -128],
[ 100, -100],
[ -37, 80],
[ 1, 0]], dtype=int8)
>>>
Python 3.13.0 (main, Oct 8 2024, 00:00:00) [GCC 14.2.1 20240912 (Red Hat 14.2.1-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> (np.array(np.array([[0, -128], [-100, 100], [37, -80], [255, 0]], np.byte))*255).astype(np.byte) # deprecation warning
<python-input-1>:1: DeprecationWarning: NumPy will stop allowing conversion of out-of-bound Python integers to integer arrays. The conversion of 255 to int8 will fail in the future.
For the old behavior, usually:
np.array(value).astype(dtype)
will give the desired result (the cast overflows).
(np.array(np.array([[0, -128], [-100, 100], [37, -80], [255, 0]], np.byte))*255).astype(np.byte) # deprecation warning
array([[ 0, -128],
[ 100, -100],
[ -37, 80],
[ 1, 0]], dtype=int8)
>>>
>>> (np.array(np.array([[0, -128], [-100, 100], [37, -80], [255, 0]], np.int16)).astype(np.byte)*255).astype(np.byte) # ok
array([[ 0, -128],
[ 100, -100],
[ -37, 80],
[ 1, 0]], dtype=int8)
>>>
I was wondering if instead of:
img_flatten = (img.flatten().astype(np.byte).astype(np.int16)*255).astype(np.byte)
it should be:
img_flatten = (img.flatten()*255).astype(np.byte)
But results could be different than today's....
@dvida , gentle ping just to make sure you are aware of this PR. I'm seeing many stations requesting numpy =>2.0 despite RMS expliciting selecting the 1.x series.
Could you try it with this code?
img_flatten = (img.ravel() * 255).astype(np.uint8).view(np.int8)