scale-clojure-web-app icon indicating copy to clipboard operation
scale-clojure-web-app copied to clipboard

Published 20 hours ago • verified publisher icon http-kit

60k+ only, how can I reach 600k !?

Open nakrakiiya opened this issue 11 years ago • 12 comments

Hi, shenfeng.

I think you can understand what I'm speaking in Chinese, if not, tell me.

我尝试了你的这个例子（包括blog上面所说的各种对系统的限制的调整方法），可从是无法达到600k。总是到60k+的时候，就开始缓慢增长，接着就不断出现这样的异常了：

java.net.ConnectException: 连接超时 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:692) at ConcurrencyBench.finishConnect(ConcurrencyBench.java:241) at ConcurrencyBench.main(ConcurrencyBench.java:150)

我用你的例子对yaws进行测试，也是同样的现象（60k+）；我该怎样调整，才能突破这个限制呢？

Ubuntu 12.04 LTS

这是使用http-kit的日志： time 0s, concurrency: 100, total requests: 0, thoughput: 0.00M/s, 0.00 requests/seconds time 1s, concurrency: 2400, total requests: 2300, thoughput: 4.43M/s, 2254.90 requests/seconds time 2s, concurrency: 5100, total requests: 5000, thoughput: 4.90M/s, 2434.27 requests/seconds time 3s, concurrency: 7800, total requests: 7700, thoughput: 5.11M/s, 2511.42 requests/seconds time 4s, concurrency: 10581, total requests: 10481, thoughput: 5.24M/s, 2575.18 requests/seconds time 5s, concurrency: 14300, total requests: 14200, thoughput: 5.69M/s, 2790.88 requests/seconds time 6s, concurrency: 17900, total requests: 17820, thoughput: 5.95M/s, 2919.88 requests/seconds time 7s, concurrency: 21400, total requests: 21389, thoughput: 6.13M/s, 3005.76 requests/seconds time 8s, concurrency: 24853, total requests: 24988, thoughput: 6.28M/s, 3071.67 requests/seconds time 9s, concurrency: 28500, total requests: 28858, thoughput: 6.45M/s, 3152.16 requests/seconds time 10s, concurrency: 32100, total requests: 32733, thoughput: 6.58M/s, 3217.95 requests/seconds time 11s, concurrency: 35593, total requests: 36636, thoughput: 6.71M/s, 3278.98 requests/seconds time 12s, concurrency: 39100, total requests: 40575, thoughput: 6.82M/s, 3331.55 requests/seconds time 13s, concurrency: 42700, total requests: 44721, thoughput: 6.93M/s, 3386.42 requests/seconds time 14s, concurrency: 46053, total requests: 48695, thoughput: 7.02M/s, 3426.57 requests/seconds time 15s, concurrency: 49600, total requests: 52906, thoughput: 7.13M/s, 3477.68 requests/seconds time 16s, concurrency: 52900, total requests: 56957, thoughput: 7.20M/s, 3512.18 requests/seconds time 17s, concurrency: 56300, total requests: 61288, thoughput: 7.30M/s, 3554.37 requests/seconds time 18s, concurrency: 59900, total requests: 65830, thoughput: 7.41M/s, 3603.17 requests/seconds time 19s, concurrency: 63400, total requests: 70405, thoughput: 7.51M/s, 3650.01 requests/seconds time 20s, concurrency: 65531, total requests: 73780, thoughput: 7.49M/s, 3632.69 requests/seconds time 21s, concurrency: 65532, total requests: 75004, thoughput: 7.24M/s, 3516.53 requests/seconds time 22s, concurrency: 65532, total requests: 76378, thoughput: 7.04M/s, 3417.97 requests/seconds time 23s, concurrency: 65532, total requests: 77774, thoughput: 6.86M/s, 3328.08 requests/seconds time 24s, concurrency: 65533, total requests: 79260, thoughput: 6.70M/s, 3252.23 requests/seconds time 25s, concurrency: 65533, total requests: 80920, thoughput: 6.57M/s, 3187.83 requests/seconds time 26s, concurrency: 65533, total requests: 82606, thoughput: 6.44M/s, 3129.96 requests/seconds time 27s, concurrency: 65533, total requests: 83898, thoughput: 6.29M/s, 3061.41 requests/seconds time 29s, concurrency: 65533, total requests: 84361, thoughput: 5.81M/s, 2819.18 requests/seconds time 34s, concurrency: 65533, total requests: 85596, thoughput: 5.07M/s, 2493.91 requests/seconds time 38s, concurrency: 65533, total requests: 85596, thoughput: 4.51M/s, 2220.39 requests/seconds time 43s, concurrency: 65533, total requests: 89332, thoughput: 4.03M/s, 2040.48 requests/seconds time 45s, concurrency: 65533, total requests: 97424, thoughput: 3.84M/s, 2118.97 requests/seconds time 47s, concurrency: 65533, total requests: 118264, thoughput: 4.28M/s, 2515.99 requests/seconds time 48s, concurrency: 65533, total requests: 122399, thoughput: 5.26M/s, 2548.92 requests/seconds time 49s, concurrency: 65533, total requests: 124303, thoughput: 5.23M/s, 2534.62 requests/seconds time 50s, concurrency: 65533, total requests: 126205, thoughput: 5.20M/s, 2521.48 requests/seconds time 51s, concurrency: 65533, total requests: 128090, thoughput: 5.17M/s, 2508.96 requests/seconds time 52s, concurrency: 65533, total requests: 129967, thoughput: 5.15M/s, 2496.68 requests/seconds time 53s, concurrency: 65533, total requests: 132509, thoughput: 5.15M/s, 2496.73 requests/seconds time 54s, concurrency: 65533, total requests: 134901, thoughput: 5.14M/s, 2493.41 requests/seconds time 55s, concurrency: 65533, total requests: 136626, thoughput: 5.10M/s, 2478.88 requests/seconds （从这里开始就是不停的“java.net.ConnectException: 连接超时”）

nakrakiiya avatar Mar 07 '13 00:03 nakrakiiya

我是朴实的中国人 :)

能告诉测试机器的情况？ OS RAM CPU JDK

其实并发数(仅)与内存有关系，内存大，就大。 一个连接http-kit 大约耗掉2k内存，OS还需要耗掉一些（TCP R/W buffer，各数K，epoll需要160bytes，等）

requests/seconds （仅）与CPU有关系。

shenfeng avatar Mar 07 '13 01:03 shenfeng

CPU: i7 920 OS: Ubuntu 12.04 LTS x64 RAM: 18G （实际上我发现64k是个坎，它甚至可以n个65532,n个65533,再n个65534，然后挂掉。所以感觉和内存无关。内存占用实际上很小，刚才观察了下，顶多4G多一点。）

java -version

java version "1.7.0_15" OpenJDK Runtime Environment (IcedTea7 2.3.7) (7u15-2.3.7-0ubuntu1~12.04) OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)

 Best regards,

Xiaofeng Yang

在 2013年3月7日上午9:01，Feng Shen [email protected]写道：

我是朴实的中国人 :)

能告诉测试机器的情况？ OS RAM CPU JDK

其实并发数(仅)与内存有关系，内存大，就大。 一个连接http-kit 大约耗掉2k内存，OS还需要耗掉一些（TCP R/W buffer，各数K，epoll需要160bytes，等）

requests/seconds （仅）与CPU有关系。

— Reply to this email directly or view it on GitHubhttps://github.com/http-kit/scale-clojure-web-app/issues/2#issuecomment-14536859 .

nakrakiiya avatar Mar 07 '13 02:03 nakrakiiya

刚才我在我这边的一台机器上试验了一下， 在连接到30～40万时，有timeout的输出，不过程序还在继续跑，数字还在增。

你的硬件应该可以达到比60万更高。

如果你感兴趣，可以试一下：

# 看看数值多少。 不太可能成问题。出了问题，服务器会报 “too many open files” 之类的错误。
ulimit -n

修改代码：

// 一次open 100连接 到服务器，然后睡20ms。 可以试着多睡一会，让服务器 多点时间处理
// 100 也可以试着调整一下。
// 你的日志 说明，好像http-kit 来不及 accept？ 
            if (opened < CONCURENCY) {
                Thread.sleep(20); // open 5000 per seconds most
            }

这是并发数：

    final static int PER_IP = 20000;
    final static InetSocketAddress ADDRS[] = new InetSocketAddress[30];
    // 600k concurrent connections
    final static int CONCURENCY = PER_IP * ADDRS.length;

你也可以试着调整一下TCP的参数。

shenfeng avatar Mar 07 '13 02:03 shenfeng

按照我现在的理解，如果调小TCP buffer，调大时间

    public static int randidelTime() {
        int ms = 5000 + r.nextInt(45000); // 5s ~ 50s
        return ms;
    }

把测试程序放到别的机器， 并发轻松上百万。

shenfeng avatar Mar 07 '13 02:03 shenfeng

你说的方法我都试过了。都不起作用。 问题的关键在于64k，也就是65535或者65536这个常数，应该是在哪个地方被限制了，所以一来熬不过这个数字，二来在接近这个数字的时候就变得增长很缓慢了。可是，这个地方又是哪里呢？ backlog其实已经设的很大，所以应该不会出现accept不过来的情况。就算如此，也不应该以一个常数作为限制呀。

ulimit -n

4999999

下面是调慢后的一个日志，后面一堆的连接超时没有被记录下来，不过在连接超时的时候数字会继续缓慢增长，但还是熬不过64k。 time 0s, concurrency: 100, total requests: 0, thoughput: 0.00M/s, 0.00 requests/seconds time 1s, concurrency: 900, total requests: 800, thoughput: 1.44M/s, 788.18 requests/seconds time 2s, concurrency: 1800, total requests: 1700, thoughput: 1.62M/s, 813.40 requests/seconds time 3s, concurrency: 2700, total requests: 2600, thoughput: 1.64M/s, 819.41 requests/seconds time 4s, concurrency: 3600, total requests: 3500, thoughput: 1.65M/s, 823.72 requests/seconds time 5s, concurrency: 4500, total requests: 4400, thoughput: 1.68M/s, 829.56 requests/seconds time 6s, concurrency: 5400, total requests: 5313, thoughput: 1.69M/s, 835.11 requests/seconds time 7s, concurrency: 6300, total requests: 6248, thoughput: 1.71M/s, 843.98 requests/seconds time 8s, concurrency: 7200, total requests: 7188, thoughput: 1.73M/s, 851.05 requests/seconds time 9s, concurrency: 8100, total requests: 8146, thoughput: 1.75M/s, 858.56 requests/seconds time 10s, concurrency: 9000, total requests: 9145, thoughput: 1.78M/s, 868.72 requests/seconds time 11s, concurrency: 9868, total requests: 10148, thoughput: 1.80M/s, 877.10 requests/seconds time 12s, concurrency: 10900, total requests: 11260, thoughput: 1.83M/s, 889.63 requests/seconds time 13s, concurrency: 11900, total requests: 12494, thoughput: 1.87M/s, 908.92 requests/seconds time 14s, concurrency: 12900, total requests: 13674, thoughput: 1.89M/s, 920.62 requests/seconds time 15s, concurrency: 13900, total requests: 14909, thoughput: 1.92M/s, 934.73 requests/seconds time 17s, concurrency: 14900, total requests: 16177, thoughput: 1.95M/s, 949.52 requests/seconds time 18s, concurrency: 15900, total requests: 17431, thoughput: 1.98M/s, 962.19 requests/seconds time 19s, concurrency: 16900, total requests: 18721, thoughput: 2.00M/s, 974.09 requests/seconds time 20s, concurrency: 17900, total requests: 20035, thoughput: 2.03M/s, 986.41 requests/seconds time 21s, concurrency: 18900, total requests: 21367, thoughput: 2.06M/s, 997.48 requests/seconds time 22s, concurrency: 19900, total requests: 22745, thoughput: 2.08M/s, 1009.99 requests/seconds time 23s, concurrency: 20900, total requests: 24172, thoughput: 2.12M/s, 1023.63 requests/seconds time 24s, concurrency: 21900, total requests: 25619, thoughput: 2.14M/s, 1036.41 requests/seconds time 25s, concurrency: 22900, total requests: 27146, thoughput: 2.17M/s, 1051.60 requests/seconds time 26s, concurrency: 23900, total requests: 28633, thoughput: 2.20M/s, 1063.95 requests/seconds time 28s, concurrency: 24900, total requests: 30165, thoughput: 2.23M/s, 1076.98 requests/seconds time 29s, concurrency: 25800, total requests: 31601, thoughput: 2.26M/s, 1089.28 requests/seconds time 30s, concurrency: 26800, total requests: 33254, thoughput: 2.28M/s, 1104.23 requests/seconds time 31s, concurrency: 27800, total requests: 34948, thoughput: 2.32M/s, 1119.20 requests/seconds time 32s, concurrency: 28800, total requests: 36655, thoughput: 2.34M/s, 1133.53 requests/seconds time 33s, concurrency: 29800, total requests: 38364, thoughput: 2.37M/s, 1147.08 requests/seconds time 34s, concurrency: 30700, total requests: 40009, thoughput: 2.40M/s, 1161.47 requests/seconds time 35s, concurrency: 31700, total requests: 41797, thoughput: 2.43M/s, 1175.56 requests/seconds time 36s, concurrency: 32700, total requests: 43619, thoughput: 2.46M/s, 1189.99 requests/seconds time 37s, concurrency: 33600, total requests: 45349, thoughput: 2.49M/s, 1204.01 requests/seconds time 38s, concurrency: 34600, total requests: 47248, thoughput: 2.52M/s, 1218.58 requests/seconds time 39s, concurrency: 35500, total requests: 49020, thoughput: 2.54M/s, 1232.18 requests/seconds time 40s, concurrency: 36400, total requests: 50866, thoughput: 2.58M/s, 1247.08 requests/seconds time 41s, concurrency: 37400, total requests: 52873, thoughput: 2.60M/s, 1261.92 requests/seconds time 42s, concurrency: 38300, total requests: 54791, thoughput: 2.64M/s, 1276.91 requests/seconds time 43s, concurrency: 39200, total requests: 56784, thoughput: 2.67M/s, 1293.07 requests/seconds time 44s, concurrency: 40100, total requests: 58738, thoughput: 2.70M/s, 1307.64 requests/seconds time 45s, concurrency: 41000, total requests: 60790, thoughput: 2.73M/s, 1323.71 requests/seconds time 46s, concurrency: 41900, total requests: 62803, thoughput: 2.76M/s, 1337.94 requests/seconds time 47s, concurrency: 42800, total requests: 64933, thoughput: 2.79M/s, 1354.21 requests/seconds time 48s, concurrency: 43700, total requests: 67060, thoughput: 2.82M/s, 1369.75 requests/seconds time 49s, concurrency: 44600, total requests: 69239, thoughput: 2.85M/s, 1385.20 requests/seconds time 50s, concurrency: 45453, total requests: 71414, thoughput: 2.89M/s, 1400.63 requests/seconds time 51s, concurrency: 46400, total requests: 73676, thoughput: 2.92M/s, 1416.98 requests/seconds time 53s, concurrency: 47300, total requests: 75958, thoughput: 2.95M/s, 1433.03 requests/seconds time 54s, concurrency: 48200, total requests: 78228, thoughput: 2.99M/s, 1448.16 requests/seconds time 55s, concurrency: 49100, total requests: 80595, thoughput: 3.02M/s, 1464.38 requests/seconds time 56s, concurrency: 50000, total requests: 82961, thoughput: 3.05M/s, 1479.76 requests/seconds time 57s, concurrency: 50900, total requests: 85381, thoughput: 3.09M/s, 1495.81 requests/seconds time 58s, concurrency: 51800, total requests: 87825, thoughput: 3.12M/s, 1511.62 requests/seconds time 59s, concurrency: 52700, total requests: 90252, thoughput: 3.15M/s, 1526.72 requests/seconds time 60s, concurrency: 53600, total requests: 92712, thoughput: 3.18M/s, 1541.47 requests/seconds time 61s, concurrency: 54500, total requests: 95272, thoughput: 3.21M/s, 1557.06 requests/seconds time 62s, concurrency: 55400, total requests: 97881, thoughput: 3.25M/s, 1573.60 requests/seconds time 63s, concurrency: 56300, total requests: 100444, thoughput: 3.28M/s, 1588.65 requests/seconds time 64s, concurrency: 57200, total requests: 103077, thoughput: 3.31M/s, 1604.44 requests/seconds time 65s, concurrency: 58100, total requests: 105749, thoughput: 3.34M/s, 1620.03 requests/seconds time 66s, concurrency: 59000, total requests: 108478, thoughput: 3.38M/s, 1636.17 requests/seconds time 67s, concurrency: 59900, total requests: 111192, thoughput: 3.41M/s, 1651.77 requests/seconds time 68s, concurrency: 60800, total requests: 113982, thoughput: 3.44M/s, 1667.77 requests/seconds time 69s, concurrency: 61700, total requests: 116785, thoughput: 3.47M/s, 1683.36 requests/seconds time 70s, concurrency: 62600, total requests: 119651, thoughput: 3.50M/s, 1699.23 requests/seconds time 71s, concurrency: 63500, total requests: 122615, thoughput: 3.54M/s, 1716.05 requests/seconds time 72s, concurrency: 64400, total requests: 125498, thoughput: 3.57M/s, 1731.58 requests/seconds time 73s, concurrency: 65300, total requests: 128346, thoughput: 3.60M/s, 1746.09 requests/seconds time 74s, concurrency: 65504, total requests: 130830, thoughput: 3.62M/s, 1753.68 requests/seconds time 75s, concurrency: 65507, total requests: 133097, thoughput: 3.63M/s, 1758.61 requests/seconds time 76s, concurrency: 65508, total requests: 135409, thoughput: 3.64M/s, 1763.96 requests/seconds time 77s, concurrency: 65510, total requests: 137721, thoughput: 3.65M/s, 1769.01 requests/seconds time 78s, concurrency: 65513, total requests: 140059, thoughput: 3.66M/s, 1774.47 requests/seconds time 80s, concurrency: 65514, total requests: 142502, thoughput: 3.68M/s, 1780.76 requests/seconds time 81s, concurrency: 65515, total requests: 144727, thoughput: 3.69M/s, 1785.28 requests/seconds time 82s, concurrency: 65515, total requests: 146872, thoughput: 3.69M/s, 1789.46 requests/seconds time 83s, concurrency: 65515, total requests: 149067, thoughput: 3.70M/s, 1793.91 requests/seconds time 84s, concurrency: 65515, total requests: 151502, thoughput: 3.72M/s, 1799.55 requests/seconds time 85s, concurrency: 65516, total requests: 153894, thoughput: 3.73M/s, 1804.76 requests/seconds time 86s, concurrency: 65518, total requests: 156283, thoughput: 3.74M/s, 1810.03 requests/seconds time 87s, concurrency: 65518, total requests: 158571, thoughput: 3.74M/s, 1815.21 requests/seconds time 88s, concurrency: 65518, total requests: 160999, thoughput: 3.76M/s, 1822.00 requests/seconds time 89s, concurrency: 65518, total requests: 163381, thoughput: 3.77M/s, 1828.06 requests/seconds time 90s, concurrency: 65518, total requests: 165730, thoughput: 3.78M/s, 1833.01 requests/seconds time 91s, concurrency: 65518, total requests: 168040, thoughput: 3.80M/s, 1838.05 requests/seconds time 92s, concurrency: 65518, total requests: 170460, thoughput: 3.80M/s, 1843.41 requests/seconds time 93s, concurrency: 65518, total requests: 172878, thoughput: 3.82M/s, 1848.69 requests/seconds time 94s, concurrency: 65518, total requests: 175248, thoughput: 3.83M/s, 1853.77 requests/seconds time 95s, concurrency: 65518, total requests: 177694, thoughput: 3.84M/s, 1859.42 requests/seconds time 96s, concurrency: 65519, total requests: 180145, thoughput: 3.85M/s, 1864.82 requests/seconds time 97s, concurrency: 65519, total requests: 182692, thoughput: 3.86M/s, 1870.73 requests/seconds time 98s, concurrency: 65519, total requests: 185321, thoughput: 3.88M/s, 1876.21 requests/seconds time 99s, concurrency: 65520, total requests: 187835, thoughput: 3.89M/s, 1881.36 requests/seconds time 100s, concurrency: 65520, total requests: 190396, thoughput: 3.90M/s, 1886.79 requests/seconds time 102s, concurrency: 65520, total requests: 193071, thoughput: 3.91M/s, 1892.61 requests/seconds time 103s, concurrency: 65520, total requests: 195581, thoughput: 3.92M/s, 1897.65 requests/seconds time 104s, concurrency: 65520, total requests: 198198, thoughput: 3.93M/s, 1903.39 requests/seconds time 105s, concurrency: 65520, total requests: 200801, thoughput: 3.94M/s, 1908.95 requests/seconds time 106s, concurrency: 65523, total requests: 203319, thoughput: 3.95M/s, 1913.73 requests/seconds time 107s, concurrency: 65523, total requests: 205807, thoughput: 3.96M/s, 1917.87 requests/seconds time 108s, concurrency: 65523, total requests: 208312, thoughput: 3.97M/s, 1922.16 requests/seconds time 109s, concurrency: 65523, total requests: 210878, thoughput: 3.98M/s, 1926.76 requests/seconds time 110s, concurrency: 65523, total requests: 213455, thoughput: 3.99M/s, 1931.44 requests/seconds time 111s, concurrency: 65524, total requests: 215986, thoughput: 4.00M/s, 1935.65 requests/seconds time 112s, concurrency: 65524, total requests: 218530, thoughput: 4.00M/s, 1939.63 requests/seconds time 113s, concurrency: 65524, total requests: 221070, thoughput: 4.01M/s, 1943.78 requests/seconds

 Best regards,

Xiaofeng Yang

2013/3/7 Feng Shen [email protected]

按照我现在的理解，如果调小TCP buffer，调大时间

public static int randidelTime() {
    int ms = 5000 + r.nextInt(45000); // 5s ~ 50s
    return ms;
}

把测试程序放到别的机器， 并发轻松上百万。

— Reply to this email directly or view it on GitHubhttps://github.com/http-kit/scale-clojure-web-app/issues/2#issuecomment-14540158 .

nakrakiiya avatar Mar 07 '13 05:03 nakrakiiya

测试程序和服务器都是我写的，没有64k的限制。 卡在64k，这么个特殊的数字，我也不解。

我用的系统是Debian，会不会是由于这个原因？ Ubuntu 和Debian 差不多的。

你很感兴趣这个问题么？ 有时间我装一个虚拟机，装Ubuntu试一下。 不过要排一段时间了。

shenfeng avatar Mar 07 '13 06:03 shenfeng

是的，很感兴趣。

非常感谢你的帮忙！

 Best regards,

Xiaofeng Yang

在 2013年3月7日下午2:10，Feng Shen [email protected]写道：

测试程序和服务器都是我写的，没有64k的限制。 卡在64k，这么个特殊的数字，我也不解。

我用的系统是Debian，会不会是由于这个原因？ Ubuntu 和Debian 差不多的。

你很感兴趣这个问题么？ 有时间我装一个虚拟机，装Ubuntu试一下。 不过要排一段时间了。

— Reply to this email directly or view it on GitHubhttps://github.com/http-kit/scale-clojure-web-app/issues/2#issuecomment-14544563 .

nakrakiiya avatar Mar 07 '13 06:03 nakrakiiya

这个问题我已经解决了。确实是被限制住了。

但引发一个新问题，web server在运行过程中，出现一堆： Fri Mar 08 22:30:41 CST 2013 [server-loop] ERROR - queue size exceeds the limit 20480, please increase :queue-size when run-server if this happens often java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6b5764b2 rejected from java.util.concurrent.ThreadPoolExecutor@7efebdb2[Running, pool size = 4, active threads = 4, queued tasks = 20478, completed tasks = 7359088] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110) at org.httpkit.server.RingHandler.handle(RingHandler.java:108) at org.httpkit.server.HttpServer.decodeHttp(HttpServer.java:114) at org.httpkit.server.HttpServer.doRead(HttpServer.java:168) at org.httpkit.server.HttpServer.run(HttpServer.java:239) at java.lang.Thread.run(Thread.java:722)

 Best regards,

Xiaofeng Yang

在 2013年3月7日下午2:10，Feng Shen [email protected]写道：

测试程序和服务器都是我写的，没有64k的限制。 卡在64k，这么个特殊的数字，我也不解。

我用的系统是Debian，会不会是由于这个原因？ Ubuntu 和Debian 差不多的。

你很感兴趣这个问题么？ 有时间我装一个虚拟机，装Ubuntu试一下。 不过要排一段时间了。

— Reply to this email directly or view it on GitHubhttps://github.com/http-kit/scale-clojure-web-app/issues/2#issuecomment-14544563 .

nakrakiiya avatar Mar 08 '13 14:03 nakrakiiya

这个问题我已经解决了

good!

确实是被限制住了。

具体？

Fri Mar 08 22:30:41 CST 2013 [server-loop] ERROR - queue size exceeds the limit 20480, please increase :queue-size when run-server if this happens often

太多请求过来，推挤了很多（超过 20480），服务器拒掉超过的请求，保护自己。 可能并发数很大时出现这个。 这个数值可设置：

(run-server handler {:port 9090 :queue-size 1024000})

shenfeng avatar Mar 08 '13 14:03 shenfeng 

 Best regards,

Xiaofeng Yang

2013/3/8 Feng Shen [email protected]

这个问题我已经解决了

good!

确实是被限制住了。

​net.netfilter.nf_conntrack_max net.ipv4.netfilter.ip_conntrack_max net.nf_conntrack_max 这三个的​值限制着这个最大连接数。修改（用sysctl）任何一个都会导致另外两个改成对应的值。把这个值调大就行了。 估计你的电脑，要么早就调整过了，要么是较老的内核（debian嘛，这个很正常）所以没有这个值。

注意一点，修改这个值最好也跟着修改/sys/module/nf_conntrack/parameters/hashsize的值，这个的值为上面那个的1/8。但是不调整这个，也能正常跑完测试。

在我的机器上，依托于强力的jvm，你的http-kit能到86W，而yaws只能到24W，而且yaws到后来的稳定性变得较差。两者最后都是内存耗尽，差别在于，使用yaws的内存增长速度比http-kit快很多。另外，yaws的CPU占用要比http-kit高一点，不过两者都在可接受范围内。

我的内核版本： $ uname -r 3.2.0-39-generic

具体？

Fri Mar 08 22:30:41 CST 2013 [server-loop] ERROR - queue size exceeds the limit 20480, please increase :queue-size when run-server if this happens often

太多请求过来，推挤了很多（超过 20480），服务器拒掉超过的请求，保护自己。 可能并发数很大时出现这个。 这个数值可设置：

(run-server handler {:port 9090 :queue-size 1024000})

— Reply to this email directly or view it on GitHubhttps://github.com/http-kit/scale-clojure-web-app/issues/2#issuecomment-14624005 .

nakrakiiya avatar Mar 09 '13 01:03 nakrakiiya

net.netfilter.nf_conntrack_max net.ipv4.netfilter.ip_conntrack_max net.nf_conntrack_max 这三个的​值限制着这个最大连接数。修改（用sysctl）任何一个都会导致另外两个改成对应的值。把这个值调大就行了。 估计你的电脑，要么早就调整过了，要么是较老的内核（debian嘛，这个很正常）所以没有这个值。

https://github.com/http-kit/scale-clojure-web-app/commit/f0be24b38bce2c9cbf041840ec6ceb386953cdc6， 别人也可能会在这里郁闷。

yaws挺不错。

最近有个Lisp的聚会： http://lisp.org.cn/wiki/lisp/event/2013-meetup 不知道你是否知道。我会去介绍一下用Clojure写Web程序，顺带介绍一下http-kit real time的支持。 还有其它几个有趣的 talk。

shenfeng avatar Mar 09 '13 01:03 shenfeng

那个聚会，我去不了。 在 2013-3-9 上午9:40，"Feng Shen" [email protected]写道：

net.netfilter.nf_conntrack_max net.ipv4.netfilter.ip_conntrack_max net.nf_conntrack_max 这三个的​值限制着这个最大连接数。修改（用sysctl）任何一个都会导致另外两个改成对应的值。把这个值调大就行了。 估计你的电脑，要么早就调整过了，要么是较老的内核（debian嘛，这个很正常）所以没有这个值。

https://github.com/http-kit/scale-clojure-web-app/commit/f0be24b38bce2c9cbf041840ec6ceb386953cdc6，别人也可能会在这里郁闷。

yaws挺不错。

最近有个Lisp的聚会： http://lisp.org.cn/wiki/lisp/event/2013-meetup 不知道你是否知道。我会去介绍一下用Clojure写Web程序，顺带介绍一下http-kit real time的支持。 还有其它几个有趣的 talk。

— Reply to this email directly or view it on GitHubhttps://github.com/http-kit/scale-clojure-web-app/issues/2#issuecomment-14655607 .

nakrakiiya avatar Mar 09 '13 01:03 nakrakiiya