ibis icon indicating copy to clipboard operation
ibis copied to clipboard

Window function causing inconsistent results

Open jjAtNxtPt opened this issue 8 months ago • 1 comments

What happened?

This is not really a bug in Ibis, but in DuckDB that is supposed to be fixed with v1.2.2 and affects Ibis. I just tested with new install of duckdb and Ibis framework and using the duckdb backend, Ibis is having this issue as well, as expected if Ibis is still using DuckDB 1.2.1.

Here are the links to both the original issue in DuckDB and my re-report because I couldn't find the original:

Original: duckdb/duckdb#16617

Mine: duckdb/duckdb#16699

Briefly: When I run a window function like min(<some_date>) over(<some_key>) using the DuckDB backend the results come back inconsistently. More specifically, taken from #16617:

When using a window function with rows between unbounded preceding and unbounded following window functions will sometimes return nulls instead of the aggregate response.

Ibis v10.3.1 is still having this issue as of this morning, probably due to not yet having DuckDB 1.2.2 integrated. It may not even be released yet.

What version of ibis are you using?

10.3.1 ibis-framework[mssql,duckdb,polars]

What backend(s) are you using, if any?

DuckDB, MSSQL, Polars

Relevant log output


Code of Conduct

  • [x] I agree to follow this project's Code of Conduct

jjAtNxtPt avatar Mar 18 '25 13:03 jjAtNxtPt

This is a great framework but now running into inconsistent results on ibis-framework 10.2.0, which is older than reported by @jjAtNxtPt. Should I go back to a certain Ibis version to get the right backend?

Is there any workaround to this bug so I can continue to use the window functions without resorting to pandas?

nickdickinson avatar May 02 '25 21:05 nickdickinson