narwhals icon indicating copy to clipboard operation
narwhals copied to clipboard

[Enh]: Support for `DataFrame.pivot`

Open FBruzzesi opened this issue 1 year ago • 1 comments

We would like to learn about your use case. For example, if this feature is needed to adopt Narwhals in an open source project, could you please enter the link to it below?

No response

Please describe the purpose of the new feature or describe the problem to solve.

pivot-al feature to add!

Suggest a solution if possible.

No response

If you have tried alternatives, please describe them below.

No response

Additional information that may help us understand your needs.

Challenges:

  • Regarding StableAPI, polars pivot would bring the first challenge in supporting two different APIs v1.0 vs v0.20
  • I am not sure that pandas-like pivot_table does support all the aggregate_functions out of the box

FBruzzesi avatar Jul 03 '24 17:07 FBruzzesi

thanks for the request, agree it would be useful

Regarding StableAPI, polars pivot would bring the first challenge in supporting two different APIs v1.0 vs v0.20

we don't need to worry about this, the idea behind the stable api is "if you write code, we won't break it" - we don't have pivot yet in Narwhals, so we don't need to introduce an old syntax. we can just add it to the v1.0 api

MarcoGorelli avatar Jul 04 '24 09:07 MarcoGorelli

Closing this as not planned for now. It never appeared in any library integration we are considering so far and the closed PR showed how inconsistent it would be to have a coherent mapping between polars versions

FBruzzesi avatar Sep 26 '24 11:09 FBruzzesi

good decision - it's usually possible to reimplement pivot in terms of groupby anyway, see the last example from https://docs.pola.rs/api/python/stable/reference/dataframe/api/polars.DataFrame.pivot.html

MarcoGorelli avatar Sep 29 '24 13:09 MarcoGorelli

looks like this would be useful to Nixtla https://github.com/Nixtla/hierarchicalforecast/pull/305

back in my data scientist days I was a huge fan of their work, they're awesome, I think we should prioritise this

MarcoGorelli avatar Nov 12 '24 17:11 MarcoGorelli

THanks for the nice words! Let me try the example, I already thought it should be possible with group_by but didn't figure it out today - the example should get me going.

Edit: the example seems problematic when using eager mode, as that will run slow, right?

elephaint avatar Nov 12 '24 21:11 elephaint