Requests beyond poolsize, and how should the pool behave

[paulscheltema]

If you do more concurrent requests than the poolSize, all requests beyond the size will be delayed by 1s

changing the minTimeout changes the delay.

1: I expected the calls to be resolved serially with the logged time ascending (each next call has to wait for the previous calls to be completed, resulting in an increasing wait time)

2: what exactly does the poolSize change? if i use apache-bench to run 1k requests 200 concurrent with a poolSize of 1 or a poolSize of 100 i see no difference in results (acutlaly the bigger poolsize seems a little bit slower, but neglectable)

3: is setting the poolSize to 1 and the minTimeout to 0 recommended? This resolves all issues with the code below. I assume the Jackpot library is used to manage loosing connections and reconnects (and also dropping/inserting servers in the hash ring), and is not used for acutal limiting/serializations of requests to memcached since the protocol is udp. (There is no actual connection to manage, just the presence of the node)

4: After doing some more apache-bench testing, it seems that using 1 server in the pool is about 40% faster with 1000+ concurrent requests than using any other number of servers (at 1000 poolSize the script just sits there doing nothing)

REPRODUCE CODE:

var memcached = require('memcached');

var memcachedClient = new memcached({'127.0.0.1:11211':1}, {poolSize: 1,minTimeout:1000});//,debug:true});

var get = function (key) { var start = (+ new Date()); console.log('start',start, key); memcachedClient.get(key, function () { var end = (+ new Date()); console.log(end-start, ' ms'); }); };

get(1);get(2);get(3);get(4);

result > start 1389015305657 1 start 1389015305666 2 start 1389015305667 3 start 1389015305667 4 13 ' ms' 1002 ' ms' 1002 ' ms' 1002 ' ms'

[arch1t3ct]

I wonder, if we are hinting on the same problem: https://github.com/3rd-Eden/node-memcached/issues/171

[paulscheltema]

Well haproxy's asci protocol runs over UDP, so there is no real connection to manage/re-use the client can just "blurp" out the information the only thing you need to manage is not to write 2 streams simultaniously, you need to serialize requests.

but! where things do need to become managed is when you use consistent hashing, then you need to know when servers go down, get back up and manage your hashring accordingly. Jackpot is used as a wrapper around the various events like disconnect, connection refused ed.

this being said, you kindof should just use one connection in total, you cannot send more requests over multiple connections than you can over a single connection, and the responses will always take longer than putting the "GET " command into a Stream buffer

the problem i ran into is that when you do more "near-concurrent" requests the memcached client goes into the pool-retry strategy waiting for the node to come back up, while that is not the problem.

long story short (and you can verify by that test script i put in my issue) using more than 1 connection in the pool slows down the total request/response times, however as it stands now you need to set the minTimeout to 0, because otherwise you will see the behaviour i reported that issue for.

also my node app is very async doing over 20 memcached lookups to generate your page and serve it in under 20ms (with a single memcached lookup taking about 10ms) so you can see that i will do many concurrent requests, not to mention when i have to handle 20 concurrent page request, that would be 20x20=> 400 concurrent memcache requests. It would be terrible to have to open 400 connections for this, so might i suggest you lower your actual poolsize. It doesnt do what you expect it to do.

2014/1/7 Andrius Virbičianskas [email protected]

I wonder, if we are hinting on the same problem: #171https://github.com/3rd-Eden/node-memcached/issues/171

— Reply to this email directly or view it on GitHubhttps://github.com/3rd-Eden/node-memcached/issues/170#issuecomment-31723089 .

groeten /paul

[paulscheltema]

Oops i just saw i saisd haproxy, i ment memcached, sorry bussy bussy

2014/1/7 Paul Scheltema [email protected]

Well haproxy's asci protocol runs over UDP, so there is no real connection to manage/re-use the client can just "blurp" out the information the only thing you need to manage is not to write 2 streams simultaniously, you need to serialize requests.

but! where things do need to become managed is when you use consistent hashing, then you need to know when servers go down, get back up and manage your hashring accordingly. Jackpot is used as a wrapper around the various events like disconnect, connection refused ed.

this being said, you kindof should just use one connection in total, you cannot send more requests over multiple connections than you can over a single connection, and the responses will always take longer than putting the "GET " command into a Stream buffer

the problem i ran into is that when you do more "near-concurrent" requests the memcached client goes into the pool-retry strategy waiting for the node to come back up, while that is not the problem.

long story short (and you can verify by that test script i put in my issue) using more than 1 connection in the pool slows down the total request/response times, however as it stands now you need to set the minTimeout to 0, because otherwise you will see the behaviour i reported that issue for.

also my node app is very async doing over 20 memcached lookups to generate your page and serve it in under 20ms (with a single memcached lookup taking about 10ms) so you can see that i will do many concurrent requests, not to mention when i have to handle 20 concurrent page request, that would be 20x20=> 400 concurrent memcache requests. It would be terrible to have to open 400 connections for this, so might i suggest you lower your actual poolsize. It doesnt do what you expect it to do.

2014/1/7 Andrius Virbičianskas [email protected]

I wonder, if we are hinting on the same problem: #171https://github.com/3rd-Eden/node-memcached/issues/171

— Reply to this email directly or view it on GitHubhttps://github.com/3rd-Eden/node-memcached/issues/170#issuecomment-31723089 .

groeten /paul

groeten /paul

[arch1t3ct]

Thanks, that clears up things quite a lot!

[paulscheltema]

Do note tho, this is how I expect it to work, and how I made it to work when i would have made the client.

However I am not yet sure this is how the author ment it to work (hence the issue)

2014/1/7 Andrius Virbičianskas [email protected]

Thanks, that clears up things quite a lot!

— Reply to this email directly or view it on GitHubhttps://github.com/3rd-Eden/node-memcached/issues/170#issuecomment-31724948 .

groeten /paul

[benperez]

This seems like a pretty huge bug in how connection pooling is handled in this library. Has this been addressed by the author of this package? It seems like anybody who uses this library for anything except the most trivial of tasks would have run into this same problem.

What really concerns me the most is this:

After doing some more apache-bench testing, it seems that using 1 server in the pool is about 40% faster with 1000+ concurrent requests than using any other number of servers

(I'm assuming he means connections in the pool, not servers) What?! Then why even have this implementation of connection pooling at all? Also, why is the default behavior of a caching library to wait 1000 ms before trying to re-connect? According to memcached documentation, it should be able to handle 200k+ requests per second on the right hardware and respond in less than a millisecond.