flink icon indicating copy to clipboard operation
flink copied to clipboard

[FLINK-32440][checkpoint] Introduce file merging configurations

Open fredia opened this issue 2 years ago • 2 comments

What is the purpose of the change

Introduce cluster-level file merging related configurations.

Brief change log

  • Add state.checkpoints.file-merging.* configurations.

Verifying this change

This change is already covered by existing tests, such as (please describe tests). ConfigOptionsYamlSpecTest.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
  • The serializers: (no)
  • The runtime per-record code paths (performance sensitive): (no)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
  • The S3 file system connector: (no)

Documentation

  • Does this pull request introduce a new feature? (yes)
  • If yes, how is the feature documented? (docs)

fredia avatar Jul 10 '23 02:07 fredia

CI report:

  • 4b30a700c8b1155e62898d963742cfbcb91d9a9c Azure: SUCCESS
Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

flinkbot avatar Jul 10 '23 02:07 flinkbot

@Zakelly Thanks for the suggestion, I have removed file merging options from docs and annotated them with @\Experimental. please take another look! Many thanks!

fredia avatar Feb 23 '24 02:02 fredia

@Zakelly I have rebased this PR, could you please take a look if you're free? thanks!

fredia avatar Apr 07 '24 12:04 fredia