aws-sdk-cpp icon indicating copy to clipboard operation
aws-sdk-cpp copied to clipboard

[aws-cpp-sdk-s3-crt] support for the exponential back-off retry strategy

Open grrtrr opened this issue 2 years ago • 5 comments

The S3CrtClient not longer has a configurable retry strategy.

By default, the S3CrtClient uses the 'standard' AWS retry strategy with default parameters (5 maximum retries, scale factor of 25msec and default/full jitter options).

In porting from S3Client to S3CrtClient deployments, support for exponential back-off is necessary, in order to retry for a longer period.

This is based on the truncated exponential backoff in https://github.com/awslabs/aws-c-io/pull/587.

Check all that applies:

  • [X] Did a review by yourself.
  • [X] Added proper tests to cover this PR. (If tests are not applicable, explain.)
  • [X] Checked if this PR is a breaking (APIs have been changed) change.
  • [X] Checked if this PR will not introduce cross-platform inconsistent behavior.
  • [X] Checked if this PR would require a ReadMe/Wiki update.

Check which platforms you have built SDK on to verify the correctness of this PR.

  • [X] Linux
  • [ ] Windows
  • [ ] Android
  • [ ] MacOS
  • [ ] IOS
  • [ ] Other Platforms

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

grrtrr avatar Aug 02 '23 16:08 grrtrr

Can you add an enum flag to opt in to this new retry strategy?

jmklix avatar Aug 14 '23 18:08 jmklix

@jmklix - how would that look like in the bigger picture of the remaining configuration? For instance, setting exponentialBackoffTotalRetries to 0 disables retries.

grrtrr avatar Aug 14 '23 19:08 grrtrr

It would look like this:

enum class AwsS3CrtRetryStrategy
  {
    DEFAULT=0, /* defaults to standard now but may change */
    STANDARD,
    EXPONENTIAL_BACKOFF,
    NO_RETRY=-1,
  };

jmklix avatar Aug 15 '23 17:08 jmklix

It would look like this:

Ok thanks. It may take a few weeks to implement this, I don't have time to get into that at the moment.

grrtrr avatar Aug 15 '23 19:08 grrtrr

@jmklix - sorry it has taken a while. I have added an enum flag as requested, specifically

  • used capital snake-case naming, for consistency with US_EAST_1_REGIONAL_ENDPOINT_OPTION in include/aws/s3-crt/S3CrtClientConfiguration.h,
  • changed parameter names to match those of the updated https://github.com/awslabs/aws-c-io/pull/587.

grrtrr avatar Dec 27 '23 20:12 grrtrr

Thanks for taking the time to make this PR, but it's no longer necessary. We added a S3 CRT retry strategy config with this PR: https://github.com/aws/aws-sdk-cpp/pull/3110. Please let us know if there are any additional things that you would like added to this sdk.

jmklix avatar Oct 04 '24 21:10 jmklix