✨ Design how to bucket rate limiting by users/requesters/etc

[roma-glushko]

The current Hyx's rate limiter APIs were inspired by resilience4j/Polly. So each instance of rate limiter handles one rate limit "shard" (e.g. basically, the level on which you apply rate limiting. It could per user, per user/request route, etc). As you could imagine, this is SUPER low level implementation which is almost certainly insufficient for the regular usecases 😌

What I have found people really needed is an API that would be integrated easily into their API framework of their choice (e.g. Flask, Starlette, FastAPI) and allowed to have many shards right away. With such implementation in place, we will close the most common and basic need for rate limiting.

Now, to be able to store a big number of shards, we will need to explore a use of probabilistic data structures, so we could "compress" the exact shard's limit quotas for the sake of a low memory footprint.

Definition of Done

• Come up with a new rate limiter API that would incorporate a need for supporting many shards. Feel free to review the existing framework-specific middleware (like https://pypi.org/project/fastapi-limiter/) to account for the step 2 of this initiative which is providing a middleware for such frameworks based on framework-agnostic rate limiters that Hyx provides
• Try to implement that new API for the tokenbucket rate limiter using probabilistic data structures to hold its shard state

References

• https://github.com/hajimes/mmh3
• https://dev.to/satrobit/rate-limiting-in-ipv6-era-using-probabilistic-data-structures-15on
• https://florian.github.io/count-min-sketch/
• https://github.com/barrust/pyprobables/blob/master/probables/hashes.py

[zentagithub]

The Tokenbucket rate limiter will work only at API level not at service level?