polars icon indicating copy to clipboard operation
polars copied to clipboard

`float` -> `Decimal` truncates Data by default (`str` -> `Decimal` does NOT)

Open Julian-J-S opened this issue 11 months ago • 0 comments

Description

When casting from float or str to Decimal without scale I see two different behaviours

  • float -> Decimal:  defaults to precision=38 & scale=0
  • str -> Decimal: defaults to precision=* and scale is determined by the data

Not sure about the "best" solution but might be good to make this consistent. I see 2 options

  1. determine scale by looking at the data
  2. use sensible defalut like scale=0 and make user explicitly specify the desired scale.
import polars as pl
pl.Config.activate_decimals(active=True)

pl.DataFrame({"f": [1.23], "s": ["1.23"]}).with_columns(
    float=pl.col("f").cast(pl.Decimal),
    str=pl.col("s").cast(pl.Decimal),
)

# shape: (1, 4)
# ┌──────┬──────┬───────────────┬──────────────┐
# │ f    ┆ s    ┆ float         ┆ str          │
# │ ---  ┆ ---  ┆ ---           ┆ ---          │
# │ f64  ┆ str  ┆ decimal[38,0] ┆ decimal[*,2] │
# ╞══════╪══════╪═══════════════╪══════════════╡
# │ 1.23 ┆ 1.23 ┆ 1             ┆ 1.23         │
# └──────┴──────┴───────────────┴──────────────┘

Julian-J-S avatar Mar 19 '24 09:03 Julian-J-S