SDK does not gracefully identify and handle throttling from S3

[ramielrowe]

Confirm by changing [ ] to [x] below to ensure that it's a bug:

• [X] I've gone though Developer Guide and API reference
• [X] I've checked AWS Forums and StackOverflow for answers
• [X] I've searched for previous similar issues and didn't find any solution

Describe the bug AWS request module doesn't consider 503 SlowDown from S3 a "Retryable" or "Throttle" error. This seemingly means that the error is unhandled by the SDK.

Version of AWS SDK for Go? Tested against 1.38.66 (latest)

Version of Go (go version)? 1.16

To Reproduce (observed behavior) https://gist.github.com/ramielrowe/613bb5bc215e69ed715cdcc96490b2b7

In the above example, I've mocked up the error we are seeing. And there are two test cases showing that it is neither a "Retryable" or "Throttle" error. I've also mocked up two errors that do pass these conditions just to confirm these tests function.

Expected behavior The S3 client library should recognize and gracefully handle 503 SlowDown errors from S3.

Additional context We've recently scaled up some of our uploading and have found to be hitting 503 SlowDown errors in our usage of MultipartUploads. We are tweaking our application to hopefully prevent this issue, but would like to add retries and backoffs as a final safe guard against issues.

[vudh1]

Hi, is this still persisting with the newest version of SDK?

[github-actions[bot]]

This issue has not received a response in 1 week. If you want to keep this issue open, please just leave a comment below and auto-close will be canceled.

[ramielrowe]

https://gist.github.com/ramielrowe/613bb5bc215e69ed715cdcc96490b2b7 has two test cases that still appear to fail on the latest release.

[ramielrowe]

@vudh1 Any update on this.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

[ramielrowe]

@vudh1 For some more context on why I believe this is particularly painful is that during large multi-part uploads, the fact that this error isn't handled by the SDK means the entire upload needs to be restarted rather than just the individual part upload that failed. This is causing us to use much much more bandwidth/request volume than necessary to complete the overall upload.

[ramielrowe]

@vudh1 Hi, have you seen my previous comment? Wondering if there's any update on this.

[lucix-aws]

This is corrected in v2.

[github-actions[bot]]

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.