narwhals icon indicating copy to clipboard operation
narwhals copied to clipboard

feat: add when-then-otherwise expression

Open aivanoved opened this issue 1 year ago • 15 comments

What type of PR is this? (check all applicable)

  • [ ] 💾 Refactor
  • [x] ✨ Feature
  • [ ] 🐛 Bug Fix
  • [ ] 🔧 Optimization
  • [ ] 📝 Documentation
  • [ ] ✅ Test
  • [ ] 🐳 Other

Related issues

  • Related issue #47

Checklist

  • [x] Code follows style guide (ruff)
  • [x] Tests added
  • [x] Documented the changes

If you have comments or can explain your changes, please do so below.

This PR so far adds a simple when-then-otherwise expression with no chaining

~The PR does not aim to be complete as of now, the current aim is to get an initial review as to if this approach is consistent with the project's quality and style of code~

TODO:

  • [x] Allow Iterable[IntoPandasExpr] to the pandas API for predicates
  • [x] ~Allow when-then-otherwise chaining~ obsolete: see comments below and #668
  • [x] Address noqa ignores

~Note: Temporary converted back to draft due to a git rebase issue~ fixed

Note: due to an unfortunate rebase gone wrong, the commit history is a mess, we could open a new pr from a new branch or when merging do a squash merge. Further advice on this would be much appreciated.

aivanoved avatar Jul 23 '24 09:07 aivanoved

@MarcoGorelli can you take another look at this?

aivanoved avatar Jul 29 '24 09:07 aivanoved

@MarcoGorelli

Note that I have added support for users to be able to provide Series or Expr as then and otherwise values, not just scalars, which I think is a crucial behaviour

Currently not all code paths are tested, according to the coverage test, am looking into this

aivanoved avatar Jul 30 '24 14:07 aivanoved

legend, thanks!

cool, once code coverage is complete we can ship it 🚀

MarcoGorelli avatar Jul 30 '24 16:07 MarcoGorelli

@MarcoGorelli

can you take another look at this?

aivanoved avatar Aug 01 '24 08:08 aivanoved

thanks @aivanoved !

looks like there's a warning in the docs build

WARNING -  griffe: narwhals/expr.py:3676: Failed to get 'name: description' pair from 'predicates'

and test failures on python 3.8

FAILED tests/expr_and_series/when_test.py::test_no_arg_when_fail[polars_eager_constructor] - ValueError: No predicates or constraints provided to `when`.
FAILED tests/expr_and_series/when_test.py::test_no_arg_when_fail[polars_lazy_constructor] - ValueError: No predicates or constraints provided to `when`.

MarcoGorelli avatar Aug 05 '24 14:08 MarcoGorelli

The doc test I will address

However, for the python 3.8 issue - the issue comes from that starting from polars version 0.20.4 onwards the error thrown when there are no predicates or constraints is a TypeError, for which we test For versions below this, meaning polars<=0.20.3, the error thrown is a ValueError, in the ci the installed version of polars is 0.20.3

I can change the error type depending on the installed polars version, however this seems unsatisfactory, please advise @MarcoGorelli

For reference here is the error thrown in version 0.20.3 And here is the same error in version 0.20.4

thanks @aivanoved !

looks like there's a warning in the docs build

WARNING -  griffe: narwhals/expr.py:3676: Failed to get 'name: description' pair from 'predicates'

and test failures on python 3.8

FAILED tests/expr_and_series/when_test.py::test_no_arg_when_fail[polars_eager_constructor] - ValueError: No predicates or constraints provided to `when`.
FAILED tests/expr_and_series/when_test.py::test_no_arg_when_fail[polars_lazy_constructor] - ValueError: No predicates or constraints provided to `when`.

aivanoved avatar Aug 05 '24 17:08 aivanoved

I think you can do with pytest.raises((ValueError, TypeError)):

any reason why you closed the PR?

MarcoGorelli avatar Aug 05 '24 17:08 MarcoGorelli

No, I guess I misclicked and canceled it by accident

aivanoved avatar Aug 05 '24 19:08 aivanoved

I think you can do with pytest.raises((ValueError, TypeError)):

any reason why you closed the PR?

As long as we are fine with having narwhals and polars throwing different types of error for polars<=0.20.3 this works, just want to make sure we are happy with the this decision

aivanoved avatar Aug 05 '24 19:08 aivanoved

@MarcoGorelli

Everything should be working now I think

aivanoved avatar Aug 06 '24 00:08 aivanoved

gentle ping @MarcoGorelli

aivanoved avatar Aug 08 '24 14:08 aivanoved

thanks for updating - hoping to merge this soon

MarcoGorelli avatar Aug 08 '24 14:08 MarcoGorelli

I'm currently on vacation will take a closer look into this again, but will be slower to respond, apologies for the delay

thank you both @MarcoGorelli and @FBruzzesi for the close review

hope we can iron everything out soon

aivanoved avatar Aug 13 '24 00:08 aivanoved

cool, no hurry, enjoy your holiday, thanks @aivanoved !

MarcoGorelli avatar Aug 13 '24 06:08 MarcoGorelli

fixed by xfail ~current ci fails with the new zip_with, due to a known issue in modin, here~

fixed ~ci fails because of this if conditioning, which is essentially a pandas version check, so the else condition does not get executed for pandas>=1.*, for me this is a tricky one to fix.failure due to coverage test~

aivanoved avatar Aug 16 '24 14:08 aivanoved

Thank you @aivanoved for working this! This is a great addition

EdAbati avatar Aug 25 '24 07:08 EdAbati