Where to store state in reqwest-middleware?

[mre]

I tried to add a new middleware for handling rate-limit headers. The idea was to use https://github.com/mre/rate-limits for extracting these headers and sleeping for a certain duration if we got rate-limited.

Even in the case of getting rate-limited, we can still return the response immediately. Only on the next request would we have to sleep until the limit got lifted. The way I thought I could implement it was to use a HashMap<String, Duration> from which I would look up the wait time for the given host. However it looks like the Middleware trait uses a non-mutable &self parameter, so I can't update the HashMap field stored in self unless I'm missing something.

The other alternative I considered was to store the HashMap inside Extensions, but that turned out to be quite unergonomic.

Am I missing anything obvious? Would love to get some feedback.

[mre]

Note: the bug label was added automatically. The issue is more of a conversation starter, so neither bug nor feature sounded right when I picked the issue template. Github Discussions might work well for such kind of issues in the future. 😉

[tl-rodrigo-gryzinski]

You're not missing anything, there's no mechanism for mutable state in the middleware data itself, since we keep the middleware stack inside Arcs we can't have a handle method that takes mutable references.

Using Extensions would work though that was intended for passing data across different middleware instances in the stack. Could wrapping the map in a Mutex or using dashmap work for your use case? Should be more ergonomic than extensions

[conradludgate]

Hi, @mre. Did you manage to solve your issue using extensions/interior mutability? I've re-labeled this issue as documentation because it would be good to have some examples of using a Mutex to update some shared state.

[mre]

Sorry, didn't find the time to look into it yet. 😕

[DerMolly]

I'm currently stuck at the exact same point.

@tl-rodrigo-gryzinski said:

Using Extensions would work though that was intended for passing data across different middleware instances in the stack.

At least in my testing the extensions seem to be cleared between calls. Maybe you juts get a different extension each time. So this could not work ever or am I missing something?

If not would it be possible to have some extra state object passed into the handle function or something similar?

[conradludgate]

The issue is that in theory you could be making a different request on many threads all at once. Any state will therefore be backed by a mutex or another concurrent structure.

I see no sensible way for us to use a mutex within the scope of reqwest-middleware for you, since they will be longer lived than your middleware can do - which is an optimisation that really matters. (ie, I'm not comfortable having us being limited to 1 request at a time, or even 1 of each middleware process since so many are stateless),

You can make a custom middleware with the state stored inside a Mutex/RwLock, or use some other concurrent data structures