Panic on data containing `Decimal("NaN")` and/or `Decimal("Infinity")`
Polars version checks
-
[X] I have checked that this issue has not already been reported.
-
[X] I have confirmed this bug exists on the latest version of Polars.
Issue description
Like float, decimal types also have NaN and Infinity values:
Reproducible example
Succeeds: (float nan/inf)
import polars as pl
s = pl.Series( [float('nan'), float('infinity')] )
# shape: (2,)
# Series: '' [f64]
# [
# NaN
# inf
# ]
Panics: (decimal nan/inf)
with pl.Config( activate_decimals=True ):
s = pl.Series( [Decimal('nan'), Decimal('infinity')] )
# thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value:
# TypeError("'str' object cannot be interpreted as an integer"),
# src/conversion.rs:727:72
Notes
There is more than one type of decimal NaN; we should recognise the following strings...
NaN,±NaN,sNAN[^1],±sNaN
...as well as:
Inf,±Inf,Infinity,±Infinity
Expected behavior
Should be able to parse/recognise decimal NaN/sNaN, and Inf/Infinity values.
Installed versions
---Version info---
Polars: 0.17.6
Index type: UInt32
Platform: macOS-13.3.1-arm64-arm-64bit
Python: 3.11.3 (main, Apr 7 2023, 20:13:31) [Clang 14.0.0 (clang-1400.0.29.202)]
---Optional dependencies---
numpy: 1.24.2
pandas: 2.0.0
pyarrow: 11.0.0
connectorx: 0.3.2_alpha.2
deltalake: 0.8.1
fsspec: <not installed>
matplotlib: <not installed>
xlsx2csv: 0.8.1
xlsxwriter: 3.0.9
[^1]: sNaN: https://mathworld.wolfram.com/SignalingNaN.html
How is Nan and infinity stored in a the i128 physical type?
Our decimal type is not one-to-one with Python's Decimal type. We currently don't support NaNs/infinities in it, and I don't expect this to ever change, leaving those strictly with the floating point types.