Rate limiting to X requests per interval

[apchamberlain]

trafficstars

Hi @jeffbski , great work on this--it's really useful!

Any best practices for rate-limiting the request flow by some wall-time measure, e.g. 100 requests per minute? For my application this would provide a more accurate simulation of real traffic--especially if for each minute the requests were scattered throughout the interval randomly rather than 100 async workers all spawning at once.

Currently I'm approximating this by putting a synchronous delay in a lambda in the the afterHooks array, but this is pretty hacky, and also messes up the stats output (since the delay time is counted towards the response time for each iteration).

If you have any suggestions for how to accomplish this with bench-rest as it stands, I'm all ears. Or if you have thoughts on how best to architect this as a new feature in the module, I'm happy to try my hand at it and send you a PR. Thanks!

[jeffbski]

@apchamberlain Thanks! I appreciate it.

I had thought about having a limit for N number of requests, but I like your idea of being able to specify rate over time. Randomizing is interesting too.

I guess we should consider what metrics make sense in this mode. Definitely latency and peak rates, but I wonder what else.

Pull requests are great, so let's figure out how we want to configure it and what we want out of it and then I can help point you in the right direction under the hood.

I'll put some thought into it and get back to you. If you have some ideas let me know as well. I look forward to working with you to add this feature.

Have a great weekend!

[apchamberlain]

If it can be done without weighing down the module with too much additional functionality that's orthogonal to its original purpose (of throwing as many requests as possible downrange at the target), I think the first step would be just to implement rate-limiting by some kind of check of a timer every time a worker is spawned (delaying as necessary without blocking anything but that worker, and tracking the overall run time against total number of requests made so far as a backup check).

The interface could be two new optional parameters in the runOptions hash:

    throttle: 30,
    interval: 'seconds',

or even just:

    throttle: { max: 30, per: 'second' },

My preference for the interaction between throttle and limit if both are specified would be that throttle would take precedence if limit is not reached by the time the maximum number of connections per interval is hit, but if limit is reached before throttle, limit is not broken. On the other hand, I can see the case for making this behavior configurable--you might want the number of simultaneous workers to adjust dynamically to keep as close as possible to a certain frequency of requests (less of a throttle than a cruise control!).

I'm still thinking about your question about metrics. Round trip time per request and all the associated statistics would still be significant. Maybe an additional histogram of deviations from the throttle value (or just max/min/median) so you can gauge how closely the desired "cruise speed" was maintained?

What do you think?

[jeffbski]

Yes, I think that is a good approach, I'm thinking about the best way to implement. I'm thinking that Rx might be a good way to throttle, but I'm still thinking through the details.