arrow
arrow copied to clipboard
GH-48457: [Python] Overflow check in datetime conversion in pa.array
Rationale for this change
https://github.com/apache/arrow/blob/7a36fcc8b7456bea52911f9601b26be51d16265a/python/pyarrow/src/arrow/python/numpy_to_arrow.cc#L499-L500
Should check and work together with safe paramter.
What changes are included in this PR?
This PR proposes to implement overflow check in date and integer conversions
Are these changes tested?
Yes, manually tested, and also added a unittest:
pytest pyarrow/tests/test_array.py -k "test_array_from_numpy_datetime_overflow" -v
Are there any user-facing changes?
Yes.
import numpy as np
import pyarrow as pa
pa.array(np.array([np.int64(-3000000000)], dtype='datetime64[D]'), safe=True)
Before:
>>> pa.array(np.array([np.int64(-3000000000)], dtype='datetime64[D]'), safe=True)
<pyarrow.lib.Date32Array object at 0x1207f85e0>
[
<value out of range: 1294967296>
]
After:
pyarrow.lib.ArrowInvalid: Integer value -3000000000 out of bounds for int32 conversion at index 0
- GitHub Issue: #48457
:warning: GitHub issue #48457 has been automatically assigned in GitHub to PR creator.