flink icon indicating copy to clipboard operation
flink copied to clipboard

[FLINK-26505][hive] Fix IndexOutOfBoundsException for non equality condition exists in left semi join in Hive dialect

Open luoyuxia opened this issue 3 years ago • 2 comments

What is the purpose of the change

The pr is try to fix the IndexOutOfBoundsException that when there's only 1 = 1 in semi join cluster like the sql: select * from a left semi join b on 1 = 1. Also, with this fix, we are also enabled to support non equality condition for left semi join which is supported in Hive HIVE-17766.

Brief change log

  • Replace the method RelOptUtil.splitJoinCondition with HiveRelOptUtil.splitHiveJoinCondition for there're some bugs in RelOptUtil.splitJoinCondition
  • Not to throw exception when there're some non-equality conditions for semi join.
  • Constuct join condition with equality conditions and non-equality conditions.

Verifying this change

Added two sql statements involved with semijoin which are to fail before this fix in join.q. select count(1) from (select x from foo where x = 1) foo1 left semi join (select i from bar where i = 1) bar2 on 1 = 1; select * from foo left semi join bar on (foo.x + bar.i > 4);

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not documented

luoyuxia avatar Mar 07 '22 09:03 luoyuxia

CI report:

  • 9dffe079c97b3d95d58ae1dbfacf1b033e04c193 Azure: SUCCESS
Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

flinkbot avatar Mar 07 '22 10:03 flinkbot

@beyond1920 @wuchong Could you please help review?

luoyuxia avatar Mar 10 '22 02:03 luoyuxia

@wuchong Could you please help review?

luoyuxia avatar Aug 16 '22 02:08 luoyuxia