boxo
boxo copied to clipboard
feat(gateway): implement request concurrency limit
This commit introduces a mechanism to limit the number of concurrent in-flight HTTP requests to the gateway.
- Implements a configurable limit for concurrent requests.
- Returns HTTP 503 Service Unavailable with Retry-After header when the limit is reached.
- Includes a default limit to protect against traffic spikes when exposed without a reverse proxy.
This change enhances gateway's resilience and prevents overload by limiting concurrent requests.
#881
Triage notes: this will be triaged as part Kubo v0.35 (@lidel )
Triage notes: moving to ~0.36~ 0.37
Triage notes: @lidel is still on it, has not forgotten about, will be dealt with in due time.
This endeavor needs more metrics and tests and integration with timeouts.
The atomic.AddInt32 proposed in this PR is ok, however feels like not the best tool for the job. Let's go with more idiomatic semaphore approach from https://github.com/ipfs/boxo/pull/994 which has lower risk of busy-waiting/checking/mutating shared int value in a tight loop.