tide

tide copied to clipboard

When servers support multiple listeners, this should be expanded to include both a global maximum as well as a per-listener maximum. This could either be a hard ceiling in which the server does not accept new connections at all until open connections are closed, or a soft ceiling at which an efficient static 503 response is sent instead of running the router.

References #577

Could also do both a soft and hard cap.

The hard limit is needed because otherwise, while you're responding with 503, your other request can be non-functional (can't open file or an upstream connection because file descriptor limit is exhausted). Having both makes sense, though.

Per-listener limit is clear how to do. With global limit it's a bit more complex. Because at the moment there is a backpressure, you have to choose which listener to get connections from first. And that might mean that other listeners are starving (in case there are priority listeners), or vice versa loosing performance on polling listeners having no connections if trying to do "fair" accept (did not benchmark if that performance difference is significant though).

I think it would also be useful if tide could respond with an HTTP 429 when it approaches some concurrency limit.

Functions like Body::from_file could also check system limits on file descriptors and respond with a 429 as it gets close to the system limit.

Thanks for reviving this issue — I forgot about it