polars icon indicating copy to clipboard operation
polars copied to clipboard

`.cumulative_eval()` in a groupby context with null values raises `PanicException`

Open cmdlineluser opened this issue 2 years ago • 0 comments

Polars version checks

  • [X] I have checked that this issue has not already been reported.

  • [X] I have confirmed this bug exists on the latest version of Polars.

Issue description

I saw this posted as a question on stackoverflow.

It doesn't appear to have been filed as a bug - so I thought I'd do it.

It seems .cumulative_eval() used in a groupby context has a problem if a group consists of all null values.

Reproducible example

import polars as pl

# Group 1 is a mix of null/non-null
df = pl.DataFrame({"group": [1, 1, 2, 3], "value": [1, None, 3, 4]})

# Okay
df.select(pl.col("value").cumulative_eval(pl.element().mean()).over("group"))
"""
shape: (4, 1)
┌───────┐
│ value │
│ ---   │
│ f64   │
╞═══════╡
│ 1.0   │
├───────┤
│ 1.0   │
├───────┤
│ 3.0   │
├───────┤
│ 4.0   │
└───────┘
"""

# Group 3 contains only nulls
df = pl.DataFrame({"group": [1, 1, 2, 3], "value": [1, None, 3, None]})

# Not Okay
df.select(pl.col("value").cumulative_eval(pl.element().mean()).over("group"))

"""
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: 
SchemaMisMatch(Borrowed("cannot unpack Series; data types don't match"))', 
/Users/user/git/polars/polars/polars-core/src/chunked_array/builder/list.rs:160:41
"""

"""
PanicException: called `Result::unwrap()` on an `Err` value:
SchemaMisMatch(Borrowed("cannot unpack Series; data types don't match"))
"""

Expected behavior

Not to raise an exception.

Installed versions

---Version info---
Polars: 0.15.16
Index type: UInt32
Python: 3.10.9 (main, Jan 14 2023, 16:56:26)
---Optional dependencies---
pyarrow: 9.0.0
pandas: 1.5.3
numpy: 1.23.5
fsspec: 2022.8.2
connectorx: <not installed>
xlsx2csv: 0.8
deltalake: <not installed>
matplotlib: 3.5.1

cmdlineluser avatar Jan 23 '23 17:01 cmdlineluser