datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

Support window functions with window frame

Open jimexist opened this issue 4 years ago • 3 comments

Is your feature request related to a problem or challenge? Please describe what you are trying to do. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] (This section helps Arrow developers understand the context and why for this feature, in addition to the what)

Implement frame spec in window clause

Describe the solution you'd like A clear and concise description of what you want to happen.

We'd like to support window function in three or more steps:

  1. #359 basic structure
  2. #298 empty over clause
  3. #299 with partition clause
  4. #360 with order by
  5. #361 with window frame (this one)

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

jimexist avatar May 19 '21 15:05 jimexist

For anyone following this, a window frame query looks like this (the ROWS BETWEEN 2 PRECEDING AND 0 FOLLOWING)

SELECT water_level, avg(water_level) OVER (ORDER BY time desc ROWS BETWEEN 2 PRECEDING AND 0 FOLLOWING)
FROM
  "h2o_feet"
WHERE
  time >= cast ('2019-09-17T20:00:00Z' as timestamp) AND
  time <= cast ('2019-09-17T21:42:00Z' as timestamp);

Which results in an error

This feature is not implemented: window expression with window frame definition is not yet supported

alamb avatar Sep 16 '22 19:09 alamb

It appears there is some work afoot to support this in https://github.com/synnada-ai/arrow-datafusion/pull/1

alamb avatar Sep 16 '22 19:09 alamb

It appears there is some work afoot to support this in synnada-ai#1

Yes, we plan to submit a PR draft early next week that makes progress on this issue. We hope to resolve it fully once we get community feedback.

ozankabak avatar Sep 18 '22 07:09 ozankabak

i am going to fix this window func issue. So we can do window func in ballista ✌️

Ted-Jiang avatar Dec 02 '22 06:12 Ted-Jiang

I have run some query success in out internal version ballista 😄 and fix some issues.

Ted-Jiang avatar Dec 02 '22 07:12 Ted-Jiang

@ozankabak Thanks again for supporting window frame.👍

Ted-Jiang avatar Dec 02 '22 07:12 Ted-Jiang

I am going to claim this ticket is done. Thank you to the Synnada team and @ozankabak and @mustafasrepo for helping make it happen 👏

alamb avatar Dec 02 '22 20:12 alamb

You are welcome, looking forward to helping in the future too!

ozankabak avatar Dec 05 '22 09:12 ozankabak