flink icon indicating copy to clipboard operation
flink copied to clipboard

[FLINK-34066][table-planner] Fix LagFunction throw NPE when input argument are not null

Open swuferhong opened this issue 1 year ago • 4 comments

What is the purpose of the change

Fix LagFunction throw NPE when input argument are not null.

This issue is related to https://issues.apache.org/jira/browse/FLINK-31967. In FLINK-31967, the NPE error has not been thoroughly fixed. If the select value LAG(len, 1, cast(null as int)) and LAG(len, 1, 1) exists together in test case AggregateITCase.testLagAggFunction() as:

val sql =
  s"""
     |select
     |  LAG(len, 1, cast(null as int)) OVER w AS nullable_prev_quantity,
     |  LAG(len, 1, 1) OVER w AS prev_quantity,
     |  LAG(len) OVER w AS prev_quantity
     |from src
     |WINDOW w AS (ORDER BY proctime)
     |""".stripMargin 

before is:

val sql =
  s"""
     |select
     |  LAG(len, 1, cast(null as int)) OVER w AS prev_quantity,
     |  LAG(len) OVER w AS prev_quantity
     |from src
     |WINDOW w AS (ORDER BY proctime)
     |""".stripMargin 

NPE will throw.

Brief change log

  • Fix LagFunction throw NPE when input argument are not null
  • Adding test to cover the npe case.

Verifying this change

  • Adding test to cover the npe case.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? no docs

swuferhong avatar Jan 12 '24 04:01 swuferhong

CI report:

  • 78c1d808d96b47cb23199d661e375ce46916fc09 Azure: SUCCESS
Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

flinkbot avatar Jan 12 '24 04:01 flinkbot

Why are we no longer testing the previous behavior?

Thanks, @MartijnVisser . I add more details about how the NPE error will be produce. Thanks for kindly remainder.

swuferhong avatar Jan 12 '24 08:01 swuferhong

@flinkbot run azure

swuferhong avatar Jan 12 '24 09:01 swuferhong

@flinkbot run azure

swuferhong avatar Jan 12 '24 10:01 swuferhong