bug: chain expression error in `ifelse` with `ibis.row_number()`

[koizumihiroo]

What happened?

Many thanks to recent upgrade to 8.0.0.

I've found a possible edge case: there's an error when using ibis.row_number() in a boolean expression, followed by an ifelse with an underscore in it. I noticed this issue in 8.0.0 but it might have been in earlier versions too.

Reproducible code

# default duckdb engin
import ibis
from ibis import _
ibis.options.interactive = True

c = ibis.examples.cars.fetch()

Error

c.mutate(dist2=(ibis.row_number()==2).ifelse(_.dist, 0))

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/me/tmp/ibis-8.0/.venv/lib/python3.11/site-packages/ibis/expr/types/logical.py", line 53, in ifelse
    return ops.IfElse(self, true_expr, false_expr).to_expr()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/tmp/ibis-8.0/.venv/lib/python3.11/site-packages/ibis/common/bases.py", line 73, in __call__
    return cls.__create__(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/tmp/ibis-8.0/.venv/lib/python3.11/site-packages/ibis/common/grounds.py", line 118, in __create__
    kwargs = cls.__signature__.validate(cls, args, kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/tmp/ibis-8.0/.venv/lib/python3.11/site-packages/ibis/common/annotations.py", line 488, in validate
    raise SignatureValidationError(
ibis.common.annotations.SignatureValidationError: <exception str() failed>

Interestingly, when running the same code in ipython (8.21.0) console, this code freezes. (no error messages)

This is ok

Boolean expresion doesn't contain ibis.row_number().

c.mutate(dist2=(_.speed==4).ifelse(_.dist, 0))

.ifelse doesn't contain underscore expression.

c.mutate(dist2=(ibis.row_number()==2).ifelse(100, 0))

Deferred mode behavior

When running with ibis.options.interactive = False, there are almost the same error messages but the last sentence differs.

ibis.options.interactive = False
c.mutate(dist2=(ibis.row_number()==2).ifelse(_.dist, 0))

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/me/tmp/ibis-8.0/.venv/lib/python3.11/site-packages/ibis/expr/types/logical.py", line 53, in ifelse
    return ops.IfElse(self, true_expr, false_expr).to_expr()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/tmp/ibis-8.0/.venv/lib/python3.11/site-packages/ibis/common/bases.py", line 73, in __call__
    return cls.__create__(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/tmp/ibis-8.0/.venv/lib/python3.11/site-packages/ibis/common/grounds.py", line 118, in __create__
    kwargs = cls.__signature__.validate(cls, args, kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/tmp/ibis-8.0/.venv/lib/python3.11/site-packages/ibis/common/annotations.py", line 488, in validate
    raise SignatureValidationError(
ibis.common.annotations.SignatureValidationError: IfElse(Equals(RowNumber(), 2): RowNumber() == 2, _.dist, 0) has failed due to the following errors:
  `true_expr`: _.dist is not coercible to a Value

Expected signature: IfElse(bool_expr: Value[Boolean, DataShape], true_expr: Value, false_null_expr: Value)

What version of ibis are you using?

macOS Sonoma 14.2.1

$ python -VV
Python 3.11.6 (main, Oct  4 2023, 17:55:52) [Clang 15.0.0 (clang-1500.0.40.1)]

$ python -c 'import ibis; print(ibis.__version__)'
8.0.0

pip list

$ pip list Package Version ------------------------ ---------- aiohttp 3.9.3 aiosignal 1.3.1 appdirs 1.4.4 asttokens 2.4.1 atpublic 4.0 attrs 23.2.0 bidict 0.22.1 cachetools 5.3.2 certifi 2024.2.2 charset-normalizer 3.3.2 decorator 5.1.1 deltalake 0.15.3 duckdb 0.9.2 duckdb_engine 0.11.1 executing 2.0.1 frozenlist 1.4.1 fsspec 2024.2.0 gcsfs 2024.2.0 google-api-core 2.17.0 google-auth 2.27.0 google-auth-oauthlib 1.2.0 google-cloud-core 2.4.1 google-cloud-storage 2.14.0 google-crc32c 1.5.0 google-resumable-media 2.7.0 googleapis-common-protos 1.62.0 greenlet 3.0.3 humanize 4.9.0 ibis-framework 8.0.0 idna 3.6 importlib-metadata 7.0.1 importlib-resources 6.1.1 ipython 8.21.0 jedi 0.19.1 Jinja2 3.1.3 joblib 1.3.2 markdown-it-py 3.0.0 MarkupSafe 2.1.5 matplotlib-inline 0.1.6 mdurl 0.1.2 multidict 6.0.5 multipledispatch 1.0.0 numpy 1.26.4 oauthlib 3.2.2 packaging 23.2 pandas 2.2.0 parso 0.8.3 parsy 2.1 pexpect 4.9.0 pins 0.8.4 pip 23.2.1 polars 0.20.7 prompt-toolkit 3.0.43 protobuf 4.25.2 ptyprocess 0.7.0 pure-eval 0.2.2 pyarrow 15.0.0 pyarrow-hotfix 0.6 pyasn1 0.5.1 pyasn1-modules 0.3.0 Pygments 2.17.2 python-dateutil 2.8.2 pytz 2024.1 PyYAML 6.0.1 regex 2023.12.25 requests 2.31.0 requests-oauthlib 1.3.1 rich 13.7.0 rsa 4.9 setuptools 65.5.0 six 1.16.0 SQLAlchemy 2.0.25 sqlalchemy-views 0.3.2 sqlglot 20.11.0 stack-data 0.6.3 toolz 0.12.1 traitlets 5.14.1 typing_extensions 4.9.0 tzdata 2023.4 urllib3 2.2.0 wcwidth 0.2.13 xxhash 3.4.1 yarl 1.9.4 zipp 3.17.

What backend(s) are you using, if any?

DuckDB

Relevant log output

No response

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[cpcloud]

Thanks for the issue and reproducible example!

I can confirm this is a problem on both main and the-epic-split (shortly merging to main, hopefully next week).

In the meantime, you can work around the problem by using the top-level ifelse function:

# instead of this
c.mutate(dist2=(ibis.row_number()==2).ifelse(_.dist, 0))

# do this
c.mutate(dist2=ibis.ifelse(ibis.row_number()==2, _.dist, 0))

[cpcloud]

It's currently not possible to mix non-deferred like this in an expression that isn't already bound to a table, like row_number().

I don't think it's possible to make that make sense, closing.