polars icon indicating copy to clipboard operation
polars copied to clipboard

`activate_decimals` Problematic and confusing

Open Julian-J-S opened this issue 11 months ago • 0 comments

Description

Activating decimal is confusing at the moment because much of the functionality is available without this but some is missing. Docs say "Activate Decimal data types."

As a user this is very confusing because

  • there is no mention of this in any error message i think
  • much functionality is there without activating first

Problems

  • Decimal functinality partly working without activation
  • No error raised! Just reverts to using floats when explicitly casting to Decimal

BEFORE activating

  • float -> decimal = float 💥 No warning/error when I explicitly want Decimal
  • str -> decimal = decimal 💥 I thought this requires "activation"?
pl.DataFrame({"x": [1.23], "s": ["1.23"]}).with_columns(
    d=pl.col("x").cast(pl.Decimal),
    d2=pl.col("s").cast(pl.Decimal(scale=1)),
)

# shape: (1, 4)
# ┌──────┬──────┬─────┬──────────────┐
# │ x    ┆ s    ┆ d   ┆ d2           │
# │ ---  ┆ ---  ┆ --- ┆ ---          │
# │ f64  ┆ str  ┆ f64 ┆ decimal[*,1] │
# ╞══════╪══════╪═════╪══════════════╡
# │ 1.23 ┆ 1.23 ┆ 1.0 ┆ 1.2          │
# └──────┴──────┴─────┴──────────────┘

AFTER activating

pl.DataFrame({"x": [1.23], "s": ["1.23"]}).with_columns(
    d=pl.col("x").cast(pl.Decimal),
    d2=pl.col("s").cast(pl.Decimal(scale=1)),
)

# shape: (1, 4)
# ┌──────┬──────┬───────────────┬──────────────┐
# │ x    ┆ s    ┆ d             ┆ d2           │
# │ ---  ┆ ---  ┆ ---           ┆ ---          │
# │ f64  ┆ str  ┆ decimal[38,0] ┆ decimal[*,1] │
# ╞══════╪══════╪═══════════════╪══════════════╡
# │ 1.23 ┆ 1.23 ┆ 1             ┆ 1.2          │
# └──────┴──────┴───────────────┴──────────────┘

Julian-J-S avatar Mar 19 '24 09:03 Julian-J-S