tower-rs
[Feature Request]: Sliding Window Ratelimit
Summary
This is a request for implementing a ratelimit layer with sliding window strategy.
Current status
The current RateLimit implemention appears to be fixed window. I.e. you can think of it like this:
- Start a clock
- Count requests while keeping an eye on the quota
- Throttle requests if the quota is up
- Reset the quota when the times up
- Repeat
(This could be documented more explicitly too. I initially believed that it would be sliding window.)
Sliding window
For quota Q and duration T, a sliding window limit guarantees that there hasn't been more than Q calls during the span [now-T, now] (the "window").
It can be thought to work like this: Keep a record of all calls for duration T. When a call comes in, it will be throttled if there's already Q calls within the window. After a while the window will have moved past those previous calls, and thus more quota will be released.
Benefits
- Tends towards less bursty traffic shape than fixed window
- Satisfies all fixed window limits of the same length and quota.
- This makes it really easy to comply with the common fixed window web API ratelimits. The client doesn't need to worry about syncing its windows with the server.
Cons
- More complex than fixed window limit
- Probably a little more overhead
Hey there! Sounds like an nice middleware. Let me suggest something: can you try it out in your own project(s) first?
We have lower maintainer attention recently, and so we can't commit to implementing large new middleware ourselves, nor reviewing and then maintaining less-tested ones. So, if you can design it locally and test it out in a few scenarios, that'd help a later submission (if you wanted to then submit it).
I understand.
Yeah I might do that some point 👍
Perhaps you could give me some pointers for the implementation? E.g. maybe some of the existing middlewares are already doing something like this?
(There does seem to be some 3rd party crates available, but none tower compatible as far as I can tell.)