datafusion
datafusion copied to clipboard
Convert `BuiltInWindowFunction::CumeDist` to a user defined window function
Is your feature request related to a problem or challenge?
Part of https://github.com/apache/datafusion/issues/8709
There is now no difference between "built in" / "prepackaged" scalar and aggregate functions in DataFusion, however there are still some "built in" window functions -- see the current source for BuiltInWindowFunction for the up to date list of what remains
The problem with having two different kinds of window functions is
- There are some features that may not be available to User Defined Window Functions that rely on built in
- Users can not easily choose which window functions to include or override the behavior if they need something different
Describe the solution you'd like
I would like to remove the "built in" version of this function and convert it to a user defined function
Describe alternatives you've considered
At a high level the process is:
- Add a new WindowUDFImpl in the functions-window crate
- Port the code from the relevant existing implementation of the the built in functions in datafusion/physical-expr/src/window
- Remove the BuiltInWindowFunction variant and then get everything to compile (the compiler will show you where the existing implementations are)
Additional context
Here are some good examples:
- Port of RowNumber in Convert built-in row_number to user-defined window function #12030
This is a good first issue.
take
@SteNicholas Just checking, do you have time to do this? If not, I can try to work on it or work on it with you in parallel.