ibis icon indicating copy to clipboard operation
ibis copied to clipboard

bug: x.isin(<contains NULLs>) result depends on backend

Open NickCrews opened this issue 1 year ago • 0 comments

What happened?

t1 = ibis.memtable({"x": [1, 2]})
e = t1.x.isin([1, None])
print(ibis.duckdb.connect().execute(e))
print(ibis.sqlite.connect().execute(e))
print(ibis.polars.connect().execute(e))
print(ibis.pandas.connect().execute(e))

yields

0 True 1 None Name: InValues(x), dtype: object 0 True 1 None Name: InValues(x), dtype: object 0 True 1 False Name: InValues(x), dtype: bool 0 True 1 None Name: InValues(x), dtype: object

If a needle isn't present in the haystack, and the haystack contains NULL, then for some backends this returns False, and for others NULL.

I think it is pretty standard SQL to return null. But this isn't what I expected to happen, I actually expected pandas semantics. IDK which we want, but they should be the same, and this should be documented.

What version of ibis are you using?

main

What backend(s) are you using, if any?

varies

Relevant log output

No response

Code of Conduct

  • [X] I agree to follow this project's Code of Conduct

NickCrews avatar Jan 23 '24 21:01 NickCrews