tower icon indicating copy to clipboard operation
tower copied to clipboard

impl `Clone` for `RateLimit <T>`

Open lcmgh opened this issue 1 year ago • 1 comments

ConcurrencyLimit <T> implements Clone here: https://docs.rs/tower/latest/src/tower/limit/concurrency/service.rs.html#96

RateLimit<T> does not https://docs.rs/tower/latest/src/tower/limit/rate/service.rs.html#14

Consequence: I am adding service specific layers by wrapping them into a new Service that auto-implements NamedService but as add_service requires clone the layer must also support it.

lcmgh avatar Sep 02 '24 09:09 lcmgh

The suggestion here is to wrap the rate-limited service inside a BufferLayer to make the resulting service clonable. This works, but adds the complexity of "running it on a background task and sending requests to it via channel". Worse, BufferLayer erases the error type, which for me is inconvenient when I wrap all this in a RetryLayer and need to make a decision as to which errors warrant retries & which do not.

@davidpdrsn noted in that comment that "If [rate-limited services] were [clonable] you'd be able to just clone them to fake being able to handle more load" and, indeed, if we were able to clone a RateLimit we could evade the rate limits.

One approach I've seen to deal with this would be to move the current RateLimit type to be non-public, maybe calling it RateLimitInner and wrap it with a new, clonable RateLimit type that holds an Arc<Mutex<RateLimitInner>>. This way we could hand out as many copies of a new RateLimit as we wanted, while not evading the rate-limiting mechanism.

I have some first code running locally... would anyone be interested in a PR for this?

sp1ff avatar May 23 '25 22:05 sp1ff