flink icon indicating copy to clipboard operation
flink copied to clipboard

[FLINK-33932][checkpointing] Add retry mechanism in RocksDBStateUploader

Open xiangyuf opened this issue 1 year ago • 3 comments

What is the purpose of the change

Add retry mechanism in RocksDBStateUploader

Brief change log

  • Introduce RetryStrategy in RocksDBStateUploader when uploadFilesToCheckpointFs

Verifying this change

This change added tests and can be verified as follows:

  • Added Unit Test

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no

xiangyuf avatar Dec 25 '23 07:12 xiangyuf

CI report:

  • 7ca00fd2aa66a369148cb97fc4a6790054e0ca0d Azure: FAILURE
Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

flinkbot avatar Dec 25 '23 07:12 flinkbot

Sorry for jumping in but may I ask about the current status?

Zakelly avatar Feb 20 '24 15:02 Zakelly

Sorry for jumping in but may I ask about the current status?

Hi @Zakelly , I'm still working on FLIP-414 to support a more general retry mechanism for all statebackends.

xiangyuf avatar Feb 21 '24 02:02 xiangyuf