flink icon indicating copy to clipboard operation
flink copied to clipboard

[FLINK-38089] Avoid pushing down non-deterministic filter in FILTER_RULES

Open Au-Miner opened this issue 5 months ago • 1 comments

What is the purpose of the change

Currently, filters with non deterministic features will be pushed down to join/agg/union, which can result in errors.

Brief change log

  • Added non deterministic check in rules in FILTER_RULE.
  • Add Test and ITCase in related operator.

Verifying this change

Existent tests and new added tests can verify this change.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? yes
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

Au-Miner avatar Jul 11 '25 09:07 Au-Miner

CI report:

  • e09be6dda1026fd1b62b15e32b965bd7d1dd2205 Azure: FAILURE
Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

flinkbot avatar Jul 11 '25 09:07 flinkbot