flink icon indicating copy to clipboard operation
flink copied to clipboard

[FLINK-26474][hive] Fold exprNode to fix the issue of failing to call some hive udf required constant parameters with implicit constant passed

Open luoyuxia opened this issue 3 years ago • 2 comments
trafficstars

What is the purpose of the change

The change is to enable ExprNode folding and to fold UDFOPNegative. It's mainly to fix the issue that it'll throw the exception BROUND second argument only takes constant when call some hive udf requiring constant parameter with implicit constant like -1 or cast(null as int) .

Brief change log

  • enable ExprNode fold
  • when the ExprNodeDesc is negative operator, try to fold it to ExprNodeConstantDesc
  • modify the logic for convertConstant in HiveParserRexNodeConverter for the value for the data type of IntervalYearMonth/IntervalDayTimeType will be class of HiveIntervalYearMonth, HiveIntervalDayTime.

Verifying this change

Added udf.q contains the sql statements that'll fail before this fix.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: (yes / no / don't know)
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? no

luoyuxia avatar Mar 04 '22 06:03 luoyuxia

CI report:

  • 6502419767b95bb0ce6e419b4f455d6e867c272d Azure: SUCCESS
Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

flinkbot avatar Mar 04 '22 06:03 flinkbot

@flinkbot run azure

luoyuxia avatar Jul 01 '22 03:07 luoyuxia

@flinkbot run azure

luoyuxia avatar Aug 22 '22 03:08 luoyuxia

@flinkbot run azure

luoyuxia avatar Aug 22 '22 04:08 luoyuxia

@flinkbot run azure

luoyuxia avatar Aug 23 '22 01:08 luoyuxia

HiveDialectQueryITCase.testCastTimeStampToDecimal:804 is failed, please take a look.

wuchong avatar Aug 26 '22 08:08 wuchong

The HiveDialectQueryITCase.testCastTimeStampToDecimal is still failed.

wuchong avatar Aug 26 '22 11:08 wuchong

@flinkbot run azure

wuchong avatar Aug 28 '22 00:08 wuchong

@flinkbot run azure

luoyuxia avatar Aug 30 '22 03:08 luoyuxia

@flinkbot run azure

luoyuxia avatar Aug 30 '22 06:08 luoyuxia

@flinkbot run azure

luoyuxia avatar Aug 30 '22 07:08 luoyuxia