add a thread pool library

[c-cube]

cc @anuragsoni that could be of interest I suppose, on multicore, directly with domains? :)

[c-cube]

It seems like this improves a bit latency and throughput, depending on the pool size. It should be better with multicore (and proper atomics!), and it's also less brutal on the OS as fewer threads are created.

Note: I'm testing with:

$ wrk -c 1000 -d 10s -t 30 --latency http://localhost:8083/hello/world

while the server is:

$ ./echo.sh -p 8083 --pool --pool-size=32

(or, for master, $ ./echo.sh -p 8083)

results

with pool (pool size=32)

$  wrk -c 1000 -d 10s -t 30 --latency http://localhost:8083/hello/world
Running 10s test @ http://localhost:8083/hello/world
  30 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   543.22us  458.20us  10.71ms   83.25%
    Req/Sec    19.69k    12.40k   40.99k    37.25%
  Latency Distribution
     50%  384.00us
     75%  696.00us
     90%    1.11ms
     99%    2.23ms
  599229 requests in 10.09s, 29.72MB read
Requests/sec:  59414.01
Transfer/sec:      2.95MB

with pool (pool size=10)

$ wrk -c 1000 -d 10s -t 30 --latency http://localhost:8083/hello/world
Running 10s test @ http://localhost:8083/hello/world
  30 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   182.29us  159.25us   7.82ms   90.12%
    Req/Sec    27.36k    16.38k   46.65k    51.47%
  Latency Distribution
     50%  133.00us
     75%  218.00us
     90%  339.00us
     99%  800.00us
  555200 requests in 10.10s, 27.53MB read
Requests/sec:  54971.09
Transfer/sec:      2.73MB

without pool

$ wrk -c 1000 -d 10s -t 30 --latency http://localhost:8083/hello/world
Running 10s test @ http://localhost:8083/hello/world
  30 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   633.62us  528.36us  11.71ms   82.82%
    Req/Sec    12.71k     8.75k   36.99k    51.60%
  Latency Distribution
     50%  463.00us
     75%  814.00us
     90%    1.29ms
     99%    2.54ms
  514955 requests in 10.07s, 25.54MB read
Requests/sec:  51137.49
Transfer/sec:      2.54MB