httpx
httpx copied to clipboard
Rate Limit Minute (-rlm) behavior does not distribute requests evenly
httpx version:
v1.6.0
Current Behavior:
When using the -rlm
(rate limit per minute) argument, the tool sends the specified amount of requests all at once instead of distributing them evenly across the minute. This behavior is counterintuitive, as it leads to a burst of traffic followed by a period of inactivity, rather than spreading the requests out to avoid overwhelming target servers.
Expected Behavior:
The expected behavior for the -rlm
argument is to distribute the specified number of requests evenly across the minute. For instance, if -rlm 10
is specified, one would expect a request to be sent every 6 seconds, thereby evenly pacing the load on the target server(s) and adhering more closely to a "rate limit."
Steps To Reproduce:
- Prepare a
urls.txt
file with multiple target URLs. - Run
httpx
with verbose logging and the-rlm
argument set to 10, like so:cat urls.txt | httpx -v -rlm 10
- Observe the output and timing of requests; all 10 requests are made at the same time, not spaced out at 1 request every 6 seconds as expected.
Anything else:
This unexpected behavior could lead to potential flooding of target websites, which is especially concerning in scenarios where careful rate limiting is necessary to comply with target server policies or to avoid unintentional Denial-of-Service conditions. An adjustment to ensure requests are distributed evenly throughout the specified time frame would greatly enhance the utility and reliability of the -rlm
feature.