Behaviour when clients disconnect

[sirupsen]

Hi, as far as I can see there's no logic to deal with clients disconnecting prematurely. Due to how the queueing mechanism is implemented, this means that if a single client disconnects all clients that came after it would loop until they time out. If you want to remove them from the queue, you'd need to use something like ngx.on_abort or the log_by_lua handler.

However, even if code ran that removed them from the queue when the client disconnected—the queue would now be in a strange state where you'd have "gaps" in the queue. This means the paused_key .. id - 1 logic would be true for more than just one entry (the front of the queue) but also for elements in the middle of the queue. So e.g. if you have 1000 clients connecting, and 500 of them disconnect and they all happen to be spaced two apart—your worst case is to let 500 clients in at once. Unless I'm missing something and the full wait loop is run even for a disconnecting client (which would raise other issues).

Additionally another potential issue I can imagine is that because you're sleeping for some amount of time, popping a large queue can take a long time—a worst case of <queue size> * <sleep time>, which could easily exceed a minute with a queue of more than 10,000 requests, and likely start rejecting requests.

What I am contemplating doing for us is to not have a queue. I don't actually care if customers get into the platform in order, all I care about is that I know exactly how many requests I let in per second at a time (leaky bucket algorithm). The current implementation of intermission.lua doesn't seem to do that, because of the first problem I described with disconnecting clients actually increasing the thundering herd.

I'm also curious about memory and CPU usage with a large amount of requests per second, is this something you've benchmarked?

@anoldguy @tweibley