ibis icon indicating copy to clipboard operation
ibis copied to clipboard

bug(ir): analytic functions are not getting dereferenced

Open kszucs opened this issue 1 year ago • 2 comments
trafficstars

What happened?

In [20]: import ibis
    ...:
    ...: t = ibis.examples.penguins.fetch()
    ...: ix = ibis.row_number().name("ix")
    ...: expr = t.mutate(ix=ix)
    ...: expr = expr.filter(ix == 5)

In [21]: expr
Out[21]:
r0 := DatabaseTable: penguins
  species           string
  island            string
  bill_length_mm    float64
  bill_depth_mm     float64
  flipper_length_mm int64
  body_mass_g       int64
  sex               string
  year              int64

r1 := Project[r0]
  species:           r0.species
  island:            r0.island
  bill_length_mm:    r0.bill_length_mm
  bill_depth_mm:     r0.bill_depth_mm
  flipper_length_mm: r0.flipper_length_mm
  body_mass_g:       r0.body_mass_g
  sex:               r0.sex
  year:              r0.year
  ix:                WindowFunction(func=RowNumber(), frame=RowsWindowFrame(r0))

Filter[r1]
  RowNumber() == 5

And RowNumber() is not valid in the filter. Dereferencing doesn't work since we transform the input row number analytic function to a window function which is different from that point.

What version of ibis are you using?

main

What backend(s) are you using, if any?

No response

Relevant log output

No response

Code of Conduct

  • [X] I agree to follow this project's Code of Conduct

kszucs avatar Feb 14 '24 15:02 kszucs

Note, that this is not a regression, with 8.0 the output is:

r0 := DatabaseTable: penguins
  species           string
  island            string
  bill_length_mm    float64
  bill_depth_mm     float64
  flipper_length_mm int64
  body_mass_g       int64
  sex               string
  year              int64

r1 := Selection[r0]
  selections:
    r0
    ix: WindowFunction(func=RowNumber(), frame=RowsWindowFrame(r0))

Selection[r1]
  predicates:
    RowNumber() == 5

kszucs avatar Feb 14 '24 17:02 kszucs

@kszucs Since this isn't a regression, let's fix this in a follow-up bugfix release.

cpcloud avatar Apr 15 '24 17:04 cpcloud

For the example here and in general, I think we can apply the same rewrite that is used during projection construction.

cpcloud avatar Aug 11 '24 13:08 cpcloud