python_backend icon indicating copy to clipboard operation
python_backend copied to clipboard

fix: python backend support for numpy>=2

Open KrishnanPrash opened this issue 1 year ago • 1 comments

Updated the PyBind11 version from v2.10.0 to v2.12.0, because v2.12 supports numpy 2 (Release Notes).

Before this update, if numpy>=2.x was installed in the same environment, performing inference requests would lead to error messages, such as:

terminate called after throwing an instance of 'pybind11::error_already_set'
  what():  error: unpack_from requires a buffer of at least 50529031 bytes for unpacking 50529027 bytes at offset 4 (actual buffer size is 7)

At:
  /opt/tritonserver/backends/python/triton_python_backend_utils.py(123): deserialize_bytes_tensor

or

File "/opt/.../py310/lib/python3.10/site-packages/tritonclient/grpc/_infer_result.py", line 93, in as_numpy
    np_array = np_array.reshape(shape)
ValueError: cannot reshape array of size 0 into shape (1,1)

KrishnanPrash avatar Oct 18 '24 01:10 KrishnanPrash

@KrishnanPrash , I still faced the issue with numpy==2.1.2 and pybind11==2.13.6. I was able to solve the issue by downgrading numpy to 1.25.1.

But shouldn't numpy=2.1.2 with pybind11=2.13.6 work as you have mentioned?

Let me know if you need more details about my code!

TheMightyRaider avatar Oct 22 '24 07:10 TheMightyRaider

Testing PR: https://github.com/triton-inference-server/server/pull/7756

rmccorm4 avatar Nov 01 '24 00:11 rmccorm4

my numpy is1.26.4, downgrading numpy to 1.25.1 is not work for me

kinglion811 avatar Feb 11 '25 14:02 kinglion811