SDK Retrier does not distinguish between service and resource throttling

[yevgenypats]

Confirm by changing [ ] to [x] below to ensure that it's a bug:

• [x] I've gone though the API reference
• [x] I've checked AWS Forums and StackOverflow for answers
• [x] I've searched for previous similar issues and didn't find any solution

Describe the bug AWS SDK is described as go-routine safe and mostly it's true as for each action all the options are copied (example).

One place that doesn't work as expected is the retrier because once you create a new service via NewFromConfig function it resolves the function resolver (i.e the client now just points to a modifiable Retryer object instead of a function).

What happens if we use the service in multiple go-routines accessing different api calls or even the same api call in different regions all those calls will share the same retrier even though throttling is per region and sometimes even per api call per region.

Version of AWS SDK for Go? v1.5.0

Version of Go (go version)? 1.16

Expected behavior I'd expect to have a fresh retrier for every invoke operation.

[jasdel]

Thanks for reaching out @yevgenypats, and taking the time to create a PR for this feature. The behavior that you're experiencing with the retryer being shared across API operation calls of the same API client instance is the intended behavior.

Then API clients are safe to use concurrently, but the API client will share some state such as retry behavior across the operation calls.

The reasoning behind this design is that errors returned by a service can include both throttling for the service or per resource. Sharing retryer across all operations of a service, ensures that the any temporary outages or service wide resources. With that said the SDK does not have the information to distinguish the difference between service wide throttling and resource specific throttling.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

[yevgenypats]

Sorry for opening / reopening - I thought I got it but I wanted to follow-up with another question.

If the SDK doesn't have a way of distinguishing between service wide throttle, region or resource specific then it basically means the best thing would be to create a client per api per region ?

[yevgenypats]

Seems we uncovered what was the issue it was client side throttling via bucket algorithm

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.