coral icon indicating copy to clipboard operation
coral copied to clipboard

[Coral-Hive] Simplify IN operator

Open wmoustafa opened this issue 2 years ago • 0 comments

What changes are proposed in this pull request, and why are they necessary?

This PR simplifies how Coral handles the expr IN (values) operator.

Previously, the following set of transformations took place: 1- Hive's ParseTreeBuilder processes the IN operator as part of visitFunctionInternal, specifically, passing the list of (expr, value_0, value_1, ...) as operands. 2- This maps to HiveInOperator when looked up from the function registry. This operator overrides createCall in such a way that makes the operands be on the format (expr, values) (i.e., instead of N operands, two operands are used with the second being the list of values). 3- HiveConvertletTable for IN operator converts IN SqlNode to a RexNode with operands on the format (expr, value_0, value_1, ...) again.

This PR sticks to the operand format (expr, value_0, value_1, ...) across the board, simplifying the back and forth transformations, and eliminating the need for the HiveConvertletTable in the IN operator case.

How was this patch tested?

  • ./gradlew build
  • Added Spark test case.
  • Will wait for integration tests.

wmoustafa avatar May 19 '23 21:05 wmoustafa