flink
flink copied to clipboard
[FLINK-38089] Avoid pushing down non-deterministic filter in FILTER_RULES
What is the purpose of the change
Currently, filters with non deterministic features will be pushed down to join/agg/union, which can result in errors.
Brief change log
- Added non deterministic check in rules in FILTER_RULE.
- Add Test and ITCase in related operator.
Verifying this change
Existent tests and new added tests can verify this change.
Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
@Public(Evolving): no - The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
CI report:
- e09be6dda1026fd1b62b15e32b965bd7d1dd2205 Azure: FAILURE
Bot commands
The @flinkbot bot supports the following commands:@flinkbot run azurere-run the last Azure build