Allow paginated operations for APIs with low throttle rates, without Rate Exceeded exceptions

[cprice404]

Describe the feature

Provide some mechanism for using the nice Flow-based Paginated APIs without running into rate exceeded exceptions on services that have low throttle limits (e.g. SSM describeInstanceInformation).

Is your Feature Request related to a problem?

Some APIs (especially describe APIs, such as SSM describeInstanceInformation) have pretty low throttling limits. When using the current Paginated kotlin functions with these APIs, users are almost guaranteed to get a Rate exceeded exception if they have multiple pages of data.

For APIs like this, a user would likely tradeoff some speed in order to stay below the throttling limit. But in the absence of some way to control this via the Paginated functions, we are forced to drop back to the non-paginated API and handle the next_tokens ourselves. A bummer since the Flow-based API for the pagination is much nicer.

Proposed Solution

Some options:

• Kotlin AWS SDK internals could catch the throttle exception and back off.
• Paginated Builders could expose some optional property like requestInterval, which, if set by the user, would cause the paginated requests to delay for a specified duration between page requests, to ensure that we stay under the throttle limit.
• Pagination logic could perhaps be aware of the throttling limits for the various services (if this information is exposed in their models), and could schedule the requests at a rate that would comply with the services' limits.

Describe alternative solutions or features you've considered

opt-out of pagination API and handle paged requests by hand.

Acknowledge

• [ ] I may be able to implement this feature request

AWS Kotlin SDK version used

v0.14.2-beta

Platform (JVM/JS/Native)

JVM

Operating System and version

Amazon Linux 2 and OSX

[aajtodd]

@cprice404

Thanks for the report. I haven't been able to reproduce this one with 35 instances running. Perhaps you're running quite a few more than this?

The paginated API's use the same underlying operation as the singular request including all of the retry middleware (which should retry throttle timeouts).

Could I ask you to grab some more debug logging here to figure out what is going on? If you add an SLF4J compatabile logger (e.g. implementation("org.slf4j:slf4j-simple:1.7.30")) and run with -Dorg.slf4j.simpleLogger.defaultLogLevel=DEBUG -Dorg.slf4j.simpleLogger.showDateTime=true as VM parameters it should log each request as well as any retries. I'd also like to see the exception (in particular what request it's happening on, how many preceded it, etc).

---

Regarding:

Paginated Builders could expose some optional property like requestInterval, which, if set by the user, would cause the paginated requests to delay for a specified duration between page requests, to ensure that we stay under the throttle limit.

You should be able to achieve this by inserting a delay manually. This isn't an optimal or recommended solution though and we want to get to the bottom of why you are having an issue. I only mention it as an FYI.

        ssm.describeInstanceInformationPaginated(req)
            .collect {
                println(it)
                println("waiting 2 seconds")
                delay(2.seconds)
            }

[cprice404]

@cprice404

Thanks for the report. I haven't been able to reproduce this one with 35 instances running. Perhaps you're running quite a few more than this?

I had about 300 instance entries in FleetManager when I was hitting this. Most of them were hybrid instances that didn't actually exist anymore, so the code that I was writing was for the purpose of looping over them all and cleaning them up... thus, I don't have them anymore and it might be hard for me to reproduce again now :(

The paginated API's use the same underlying operation as the singular request including all of the retry middleware (which should retry throttle timeouts).

Could I ask you to grab some more debug logging here to figure out what is going on? If you add an SLF4J compatabile logger (e.g. implementation("org.slf4j:slf4j-simple:1.7.30")) and run with -Dorg.slf4j.simpleLogger.defaultLogLevel=DEBUG -Dorg.slf4j.simpleLogger.showDateTime=true as VM parameters it should log each request as well as any retries. I'd also like to see the exception (in particular what request it's happening on, how many preceded it, etc).

yup we are using logback so if I hit this again anywhere else, I'll be sure to capture the debug logs.

Regarding:

Paginated Builders could expose some optional property like requestInterval, which, if set by the user, would cause the paginated requests to delay for a specified duration between page requests, to ensure that we stay under the throttle limit.

You should be able to achieve this by inserting a delay manually. This isn't an optimal or recommended solution though and we want to get to the bottom of why you are having an issue. I only mention it as an FYI.

        ssm.describeInstanceInformationPaginated(req)
            .collect {
                println(it)
                println("waiting 2 seconds")
                delay(2.seconds)
            }

Thanks. I think this would cause a delay to happen between every instance, rather than every page, so as you said, not optimal... but it is an idea that I hadn't thought of :)

[aajtodd]

Ok I see. I'm going to leave this open for now then and see if anyone else runs into something similar. My concern is that something is off in the retry middleware but hard to say without more information.

---

I think this would cause a delay to happen between every instance, rather than every page

describeInstanceInformationPaginated() is the original paged response so the delay would be per/page. If you use the extension function to get at the "items" of the paginator (instanceInformationList()) then yes you are correct it would be on every instance then like so:

ssm.describeInstanceInformationPaginated()
       .instanceInformationList()
       .collect { instanceInfo -> 
            // individual instance
        }

[aajtodd]

Closing this as not actionable. Please open a new ticket as necessary.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.