ibis
ibis copied to clipboard
bug(ir): analytic functions are not getting dereferenced
trafficstars
What happened?
In [20]: import ibis
...:
...: t = ibis.examples.penguins.fetch()
...: ix = ibis.row_number().name("ix")
...: expr = t.mutate(ix=ix)
...: expr = expr.filter(ix == 5)
In [21]: expr
Out[21]:
r0 := DatabaseTable: penguins
species string
island string
bill_length_mm float64
bill_depth_mm float64
flipper_length_mm int64
body_mass_g int64
sex string
year int64
r1 := Project[r0]
species: r0.species
island: r0.island
bill_length_mm: r0.bill_length_mm
bill_depth_mm: r0.bill_depth_mm
flipper_length_mm: r0.flipper_length_mm
body_mass_g: r0.body_mass_g
sex: r0.sex
year: r0.year
ix: WindowFunction(func=RowNumber(), frame=RowsWindowFrame(r0))
Filter[r1]
RowNumber() == 5
And RowNumber() is not valid in the filter. Dereferencing doesn't work since we transform the input row number analytic function to a window function which is different from that point.
What version of ibis are you using?
main
What backend(s) are you using, if any?
No response
Relevant log output
No response
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
Note, that this is not a regression, with 8.0 the output is:
r0 := DatabaseTable: penguins
species string
island string
bill_length_mm float64
bill_depth_mm float64
flipper_length_mm int64
body_mass_g int64
sex string
year int64
r1 := Selection[r0]
selections:
r0
ix: WindowFunction(func=RowNumber(), frame=RowsWindowFrame(r0))
Selection[r1]
predicates:
RowNumber() == 5
@kszucs Since this isn't a regression, let's fix this in a follow-up bugfix release.
For the example here and in general, I think we can apply the same rewrite that is used during projection construction.