flink icon indicating copy to clipboard operation
flink copied to clipboard

Published 20 hours ago • verified publisher icon apache

[FLINK-34615]Split `ExternalizedCheckpointCleanup` out of `Checkpoint…

Open spoon-lz opened this issue 11 months ago • 17 comments

…Config` and move it to flink-core.

What is the purpose of the change

In FLIP-406 we want to merge all options from ExecutionCheckpointingOptions into CheckpointingOptions. We need to split ExecutionCheckpointingOptions from CheckpointConfig and move it to flink-core. This PR introduces a new ExecutionCheckpointingOptions in flink-core and deprecates the old CheckpointConfig.ExecutionCheckpointingOptions and corresponding user-facing API.

Verifying this change

  • introduce a new ExecutionCheckpointingOptions in flink-core, providing new APIs.
  • deprecate the old CheckpointConfig.ExecutionCheckpointingOptions in flink-streaming-java as well as the APIs.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (yes / no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
  • The serializers: (yes / no / don't know)
  • The runtime per-record code paths (performance sensitive): (yes / no / don't know)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
  • The S3 file system connector: (yes / no / don't know)

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

spoon-lz avatar Mar 07 '24 08:03 spoon-lz

CI report:

  • 1927210608564f68ded88588081908674493ed42 UNKNOWN
  • 9cf67cdf47f2230c0fc7c05ce0b2011a7ce3b7c4 Azure: SUCCESS
Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

flinkbot avatar Mar 07 '24 08:03 flinkbot

@Zakelly Please help me take a look at this PR to see if it meets the requirements.

spoon-lz avatar Mar 08 '24 02:03 spoon-lz

@flinkbot run azure

spoon-lz avatar Mar 14 '24 09:03 spoon-lz

@Zakelly The code has been modified, please take a look again

spoon-lz avatar Mar 15 '24 08:03 spoon-lz

@flinkbot run azure

spoon-lz avatar Apr 02 '24 02:04 spoon-lz

@flinkbot run azure

spoon-lz avatar Apr 02 '24 07:04 spoon-lz

@flinkbot run azure

spoon-lz avatar Apr 02 '24 08:04 spoon-lz

@Zakelly @masteryhx The code has been adjusted, please help me take a look again.

spoon-lz avatar Apr 02 '24 10:04 spoon-lz

@masteryhx Hi,this pr blocks another pr. Can you help me take a look? By the way, this change caused a python test error. Since I don’t have much experience in using python, I modified it several times and failed the test.

spoon-lz avatar Apr 10 '24 09:04 spoon-lz

@masteryhx Hi,this pr blocks another pr. Can you help me take a look? By the way, this change caused a python test error. Since I don’t have much experience in using python, I modified it several times and failed the test.

It seems the CI pipeline failed to trigger test for your last commit. How about re-commit your last commit and force push to trigger the CI?

Zakelly avatar Apr 11 '24 02:04 Zakelly

@flinkbot run azure

masteryhx avatar Apr 11 '24 02:04 masteryhx

@flinkbot run azure

spoon-lz avatar Apr 11 '24 09:04 spoon-lz

@Zakelly @masteryhx What further discussions are needed on the modification of the python part? This PR has been on hold for some time.

spoon-lz avatar May 07 '24 02:05 spoon-lz

@spoon-lz sorry for the late reply. I agreed with @masteryhx that we'd better create another new class and deprecate the old one in python.

Zakelly avatar May 09 '24 07:05 Zakelly

@flinkbot run azure

spoon-lz avatar May 11 '24 08:05 spoon-lz

@Zakelly @masteryhx A new ExternalizedCheckpointRetention.py is created, the old one is marked as deprecated, and a separate commit for the python part is split.

spoon-lz avatar May 13 '24 02:05 spoon-lz

@flinkbot run azure

spoon-lz avatar May 15 '24 07:05 spoon-lz

@Zakelly Flink-ci has passed

spoon-lz avatar May 17 '24 06:05 spoon-lz

I modified the commit message and remove those 'cherry-picked from xxxxx'. Already merged into master.

Zakelly avatar May 17 '24 07:05 Zakelly