flink
flink copied to clipboard
[FLINK-33986][runtime] Extend ShuffleMaster to support snapshot and restore state.
What is the purpose of the change
Extend ShuffleMaster to support snapshot and restore state.
Brief change log
Extend shuffleMaster to support batch snapshot as follows:
- Add method supportsBatchSnapshot to identify whether the shuffle master supports taking snapshot in batch scenarios
- Add method snapshotState and restoreState to snapshot and restore the shuffle master's state.
Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / no)
- The public API, i.e., is any changed class annotated with
@Public(Evolving)
: (yes / no) - The serializers: (yes / no / don't know)
- The runtime per-record code paths (performance sensitive): (yes / no / don't know)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
- The S3 file system connector: (yes / no / don't know)
Documentation
- Does this pull request introduce a new feature? (yes / no)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
CI report:
- 0a268a5463696e20e06e05b5bd6355d8663455f8 Azure: SUCCESS
Bot commands
The @flinkbot bot supports the following commands:-
@flinkbot run azure
re-run the last Azure build
Thanks @zhuzhurk for review, I've updated this pr accordingly. PTAL~