ibis icon indicating copy to clipboard operation
ibis copied to clipboard

feat(flink): support semi/anti window join

Open mfatihaktas opened this issue 1 year ago • 0 comments

Description of changes

Aims to close https://github.com/ibis-project/ibis/issues/8710.

Notes to reviewers:

  • Adds AntiWindowJoin and SemiWindowJoin sqlglot expressions. Would there be a way to avoid these? I could not find an existing sqlglot expression that we can use.
  • Adds two new entries in JoinKind: anti_window and semi_window. Added a TODO with a question on if we can detect a join to be window join by looking for WindowingTVF in the expression tress for left and right. However, that "auto-window-join-detect" might lead to unexpected behavior, hence chose to make window-joins explicit with new JoinKind entries.

Edit: Got feedback from @kszucs over Zulip, and refactored the previously added ops and expressions mentioned above. Now instead, JoinChain is re-written with Filter and ExistsSubquery by FlinkCompiler.

mfatihaktas avatar Mar 22 '24 16:03 mfatihaktas