python_backend
python_backend copied to clipboard
fix: python backend support for numpy>=2
Updated the PyBind11 version from v2.10.0 to v2.12.0, because v2.12 supports numpy 2 (Release Notes).
Before this update, if numpy>=2.x was installed in the same environment, performing inference requests would lead to error messages, such as:
terminate called after throwing an instance of 'pybind11::error_already_set'
what(): error: unpack_from requires a buffer of at least 50529031 bytes for unpacking 50529027 bytes at offset 4 (actual buffer size is 7)
At:
/opt/tritonserver/backends/python/triton_python_backend_utils.py(123): deserialize_bytes_tensor
or
File "/opt/.../py310/lib/python3.10/site-packages/tritonclient/grpc/_infer_result.py", line 93, in as_numpy
np_array = np_array.reshape(shape)
ValueError: cannot reshape array of size 0 into shape (1,1)
@KrishnanPrash , I still faced the issue with numpy==2.1.2 and pybind11==2.13.6. I was able to solve the issue by downgrading numpy to 1.25.1.
But shouldn't numpy=2.1.2 with pybind11=2.13.6 work as you have mentioned?
Let me know if you need more details about my code!
Testing PR: https://github.com/triton-inference-server/server/pull/7756
my numpy is1.26.4, downgrading numpy to 1.25.1 is not work for me