frp 长时间运行和多连接积累时,会无法处理指定端口

[REQUIRED] hat version of frp are you using

Version:

[REQUIRED] What operating system and processor architecture are you using OS: Linux amd64 CPU architecture: Intel 实际上我是在群晖上面跑的FRPS端.他一切正常,使用的版本0.34.x这个版本(就35发布前的版本都有运行过),运行起来一切正常,问题在于运行一段时间后.

[REQUIRED] description of errors

confile

log file

Steps to reproduce the issue

我在内网开了一个端口80,使用FRP映射内网的这个机器(192.168.1.100:80)出来. FRPS上面映射的端口是8000.
运行了两周时间,一切正常. 昨天晚上突然发现8000这个端口无法再接受请求了. 表现是FRPS对8000端口的请求,都是统一的只接受,不应答. 3.FRPS上面映射的其他端口是正常的. 仅8000端口出现了问题. 解决办法:我重启了FRPS,恢复正常.

Supplementary information

Can you guess what caused this issue 我觉得这可能是FRPS在处理来自8000的端口时,有一个端口接受上限, 是做的TCP转发,实际上是HTTP协议过来的数据 ,建立连接后并且保持了连接,长时间运行,导致可连接到8000的客户端达到了上限. 请问是否有处理建议? 我现在的做法是每周重启一次FRPS,这样可以避免这个问题. Checklist:

[] I included all information required in the sections above
[] I made sure there are no duplicates of this report (Use Search)

Jan 22 '21 01:01 zsinba

我猜想,是不是FRPS里,对一个端口的连接,做了上限限制,导致达到这个上限后,就不再处理. 但是表现并不太像,我查看了(netstat -ano >1.log)端口连接情况,并不是很多端口,系统所有连接加起来,也才一千左右. 没有其他办法,我只能重启FRPS,然后访问就恢复正常了. 所以问题应该还是在FRPS上面. FRPC我没有去动它.

Jan 22 '21 01:01 zsinba

可能需要更多点的信息：

“只接受,不应答“ 是否 frps 有哪些日志
结合 frps dashboard 看看有没有信息，比如说 proxy connections
能不能提供一个复现的场景或者思路

Jan 22 '21 02:01 yuyulei

可能需要更多点的信息：

“只接受,不应答“ 是否 frps 有哪些日志日志可能用途不太大，没看出来异常。如下日志。

结合 frps dashboard 看看有没有信息，比如说 proxy connections 没有异常，正常的连接状态的。

能不能提供一个复现的场景或者思路我一共出现了两次，每次间隔时间大概两周。每次都是重启frps解决的

Jan 22 '21 02:01 zsinba

可能需要更多点的信息：

“只接受,不应答“ 是否 frps 有哪些日志日志可能用途不太大，没看出来异常。如下日志。

结合 frps dashboard 看看有没有信息，比如说 proxy connections 没有异常，正常的连接状态的。

能不能提供一个复现的场景或者思路我一共出现了两次，每次间隔时间大概两周。每次都是重启frps解决的

日志在这里： 2021/01/21 23:05:47 [I] [proxy.go:103] [5043827614628e93] [YYYYYYYYYYYYRDP] get a new work connection: [11.11.11.11:61100]

2021/01/21 23:07:33 [I] [proxy.go:159] [b3c99ad587e6a41c] [XXXXXXXXXHHHHHHH] listener is closed 2021/01/21 23:07:33 [I] [proxy.go:103] [b3c99ad587e6a41c] [XXXXXXXXXHHHHHHH] get a new work connection: [192.168.1.159:49671] 2021/01/21 23:07:33 [I] [proxy.go:159] [b3c99ad587e6a41c] [XXXXXXXXXXYYver] listener is closed 2021/01/21 23:07:34 [I] [proxy.go:159] [c4c92ff580cef8ce] [XXXXY_YYY] listener is closed 2021/01/21 23:07:34 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXY_YYY] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:07:35 [I] [proxy.go:159] [5043827614628e93] [YYYYYYYYYYYYRDP] listener is closed 2021/01/21 23:07:35 [I] [proxy.go:103] [5043827614628e93] [YYYYYYYYYYYYRDP] get a new work connection: [11.11.11.11:61100] 2021/01/21 23:07:51 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:08:37 [I] [proxy.go:103] [b3c99ad587e6a41c] [EEEEEEEEEE2XXXYYver] get a new work connection: [192.168.1.159:49671] 2021/01/21 23:09:11 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:10:33 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:11:59 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:13:18 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:13:57 [I] [proxy.go:103] [b3c99ad587e6a41c] [EEEEEEEEEE2XXXYYver] get a new work connection: [192.168.1.159:49671] 2021/01/21 23:14:02 [I] [proxy.go:103] [eab1f2be0c69e775] [666666665.21777778020] get a new work connection: [192.168.3.212:46942] 2021/01/21 23:14:40 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:16:07 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:17:28 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:18:48 [W] [vhost.go:147] get hostname from http/https request error: read tcp 192.168.3.192:1443->89.248.165.94:2168: i/o timeout 2021/01/21 23:18:55 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:19:00 [I] [dashboard_api.go:65] Http request: [/api/serverinfo] 2021/01/21 23:19:00 [I] [dashboard_api.go:58] Http response [/api/serverinfo]: code [200] 2021/01/21 23:19:10 [I] [dashboard_api.go:180] Http request: [/api/proxy/tcp] 2021/01/21 23:19:10 [I] [dashboard_api.go:174] Http response [/api/proxy/tcp]: code [200] 2021/01/21 23:19:17 [I] [proxy.go:103] [b3c99ad587e6a41c] [EEEEEEEEEE2XXXYYver] get a new work connection: [192.168.1.159:49671] 2021/01/21 23:19:18 [I] [dashboard_api.go:180] Http request: [/api/proxy/tcp] 2021/01/21 23:19:18 [I] [dashboard_api.go:174] Http response [/api/proxy/tcp]: code [200] 2021/01/21 23:19:24 [I] [dashboard_api.go:180] Http request: [/api/proxy/tcp] 2021/01/21 23:19:24 [I] [dashboard_api.go:174] Http response [/api/proxy/tcp]: code [200] 2021/01/21 23:19:25 [I] [dashboard_api.go:180] Http request: [/api/proxy/tcp] 2021/01/21 23:19:25 [I] [dashboard_api.go:174] Http response [/api/proxy/tcp]: code [200] 2021/01/21 23:19:25 [I] [dashboard_api.go:180] Http request: [/api/proxy/tcp] 2021/01/21 23:19:25 [I] [dashboard_api.go:174] Http response [/api/proxy/tcp]: code [200] 2021/01/21 23:19:26 [I] [dashboard_api.go:180] Http request: [/api/proxy/tcp] 2021/01/21 23:19:26 [I] [dashboard_api.go:174] Http response [/api/proxy/tcp]: code [200] 2021/01/21 23:19:26 [I] [dashboard_api.go:180] Http request: [/api/proxy/tcp] 2021/01/21 23:19:26 [I] [dashboard_api.go:174] Http response [/api/proxy/tcp]: code [200] 2021/01/21 23:19:26 [I] [dashboard_api.go:180] Http request: [/api/proxy/tcp] 2021/01/21 23:19:26 [I] [dashboard_api.go:174] Http response [/api/proxy/tcp]: code [200] 2021/01/21 23:19:56 [I] [proxy.go:103] [b3c99ad587e6a41c] [EEEEEEEEEE2XXXYYver] get a new work connection: [192.168.1.159:49671] 2021/01/21 23:19:58 [I] [proxy.go:103] [eab1f2be0c69e775] [666666665.2177777122] get a new work connection: [192.168.3.212:46942] 2021/01/21 23:20:42 [I] [proxy.go:103] [b3c99ad587e6a41c] [DDDDDD59server] get a new work connection: [192.168.1.159:49671] 2021/01/21 23:20:43 [I] [proxy.go:103] [eab1f2be0c69e775] [666666665.21777779000] get a new work connection: [192.168.3.212:46942] 2021/01/21 23:20:43 [I] [proxy.go:103] [eab1f2be0c69e775] [666666665.21777779000] get a new work connection: [192.168.3.212:46942] 2021/01/21 23:21:39 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:23:07 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:24:34 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:24:37 [I] [proxy.go:103] [b3c99ad587e6a41c] [EEEEEEEEEE2XXXYYver] get a new work connection: [192.168.1.159:49671] 2021/01/21 23:25:59 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:27:26 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:28:52 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:29:04 [I] [dashboard_api.go:180] Http request: [/api/proxy/tcp] 2021/01/21 23:29:04 [I] [dashboard_api.go:174] Http response [/api/proxy/tcp]: code [200]

2021/01/21 23:29:57 [I] [proxy.go:103] [b3c99ad587e6a41c] [EEEEEEEEEE2XXXYYver] get a new work connection: [192.168.1.159:49671] 2021/01/21 23:30:01 [I] [proxy.go:103] [b3c99ad587e6a41c] [666666665.sbin_vm] get a new work connection: [192.168.1.159:49671] 2021/01/21 23:30:04 [I] [proxy.go:103] [b3c99ad587e6a41c] [666666665.sbin_vm] get a new work connection: [192.168.1.159:49671] 2021/01/21 23:30:20 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:31:25 [I] [dashboard_api.go:180] Http request: [/api/proxy/tcp] 2021/01/21 23:31:25 [I] [dashboard_api.go:174] Http response [/api/proxy/tcp]: code [200] 2021/01/21 23:31:33 [I] [dashboard_api.go:174] Http response [/api/proxy/tcp]: code [200] 2021/01/21 23:31:51 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:33:17 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:33:18 [I] [proxy.go:103] [b3c99ad587e6a41c] [DDDDDD59server] get a new work connection: [192.168.1.159:49671] 2021/01/21 23:34:51 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:35:17 [I] [proxy.go:103] [b3c99ad587e6a41c] [EEEEEEEEEE2XXXYYver] get a new work connection: [192.168.1.159:49671] 2021/01/21 23:36:16 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:37:00 [I] [dashboard_api.go:314] Http request: [/api/traffic/XXXXXXXXXXYYver] 2021/01/21 23:37:00 [I] [dashboard_api.go:308] Http response [/api/traffic/XXXXXXXXXXYYver]: code [200] 2021/01/21 23:37:01 [I] [dashboard_api.go:180] Http request: [/api/proxy/tcp] 2021/01/21 23:37:01 [I] [dashboard_api.go:174] Http response [/api/proxy/tcp]: code [200] 2021/01/21 23:37:02 [I] [dashboard_api.go:180] Http request: [/api/proxy/tcp] 2021/01/21 23:37:02 [I] [dashboard_api.go:174] Http response [/api/proxy/tcp]: code [200] 2021/01/21 23:37:02 [I] [dashboard_api.go:180] Http request: [/api/proxy/tcp] 2021/01/21 23:37:02 [I] [dashboard_api.go:174] Http response [/api/proxy/tcp]: code [200] 2021/01/21 23:37:03 [I] [dashboard_api.go:180] Http request: [/api/proxy/tcp] 2021/01/21 23:37:03 [I] [dashboard_api.go:174] Http response [/api/proxy/tcp]: code [200] 2021/01/21 23:37:03 [I] [dashboard_api.go:314] Http request: [/api/traffic/XXXXXXXXXXYYver] 2021/01/21 23:37:03 [I] [dashboard_api.go:308] Http response [/api/traffic/XXXXXXXXXXYYver]: code [200] 2021/01/21 23:37:18 [I] [proxy.go:103] [b3c99ad587e6a41c] [DDDDDD59server] get a new work connection: [192.168.1.159:49671] 2021/01/21 23:37:46 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:39:12 [I] [proxy.go:103] [c4c92ff580cef8ce] [XXXXDDWF] get a new work connection: [192.168.3.212:41584] 2021/01/21 23:40:00 [I] [control.go:309] [c4c92ff580cef8ce] control writer is closing 2021/01/21 23:40:00 [I] [proxy.go:87] [c4c92ff580cef8ce] [XXXXY_YYY] proxy closing 2021/01/21 23:40:00 [I] [proxy.go:87] [c4c92ff580cef8ce] [XXXXDDWF] proxy closing 2021/01/21 23:40:00 [I] [proxy.go:159] [c4c92ff580cef8ce] [XXXXDDWF] listener is closed 2021/01/21 23:40:00 [I] [proxy.go:87] [c4c92ff580cef8ce] [XXXXXXXXX_ROUTER] proxy closing 2021/01/21 23:40:00 [I] [control.go:384] [c4c92ff580cef8ce] client exit success 2021/01/21 23:40:01 [I] [service.go:444] [11061fbdcabb6762] client login info: ip [192.168.3.212:42006] version [0.30.0] hostname [] os [linux] arch [arm] 2021/01/21 23:40:01 [I] [tcp.go:63] [11061fbdcabb6762] [XXXXY_YYY] tcp proxy listen port [8895] 2021/01/21 23:40:01 [I] [control.go:446] [11061fbdcabb6762] new proxy [XXXXY_YYY] success 2021/01/21 23:40:01 [I] [http.go:124] [11061fbdcabb6762] [XXXXXXXXX_ROUTER] http proxy listen for host [pc.jqk.cool] location [] group [] 2021/01/21 23:40:01 [I] [control.go:446] [11061fbdcabb6762] new proxy [XXXXXXXXX_ROUTER] success 2021/01/21 23:40:01 [I] [tcp.go:63] [11061fbdcabb6762] [XXXXDDWF] tcp proxy listen port [8892] 2021/01/21 23:40:01 [I] [control.go:446] [11061fbdcabb6762] new proxy [XXXXDDWF] success 2021/01/21 23:40:01 [I] [proxy.go:103] [b3c99ad587e6a41c] [DDDDDDDDXXXYYver] get a new work connection: [192.168.1.159:49671] 2021/01/21 23:40:01 [I] [proxy.go:103] [11061fbdcabb6762] [XXXXY_YYY] get a new work connection: [192.168.3.212:42006] 2021/01/21 23:40:04 [I] [proxy.go:103] [11061fbdcabb6762] [XXXXY_YYY] get a new work connection: [192.168.3.212:42006]

2021/01/21 23:42:37 [I] [proxy.go:103] [11061fbdcabb6762] [XXXXY_YYY] get a new work connection: [192.168.3.212:42006] 2021/01/21 23:42:41 [I] [service.go:190] frps tcp listen on 0.0.0.0:18080 2021/01/21 23:42:41 [I] [service.go:199] frps kcp listen on udp 0.0.0.0:18080 2021/01/21 23:42:41 [I] [service.go:232] http service listen on 0.0.0.0:18080 2021/01/21 23:42:41 [I] [service.go:253] https service listen on 0.0.0.0:1443 2021/01/21 23:42:41 [I] [service.go:271] nat hole udp service listen on 0.0.0.0:7001 2021/01/21 23:42:41 [I] [service.go:289] Dashboard listen on 0.0.0.0:17500 2021/01/21 23:42:41 [I] [root.go:215] start frps success 2021/01/21 23:42:41 [I] [service.go:444] [bfd8adec45a196d0] client login info: ip [192.168.1.233:58165] version [0.34.1] hostname [] os [windows] arch [amd64] 2021/01/21 23:42:41 [I] [tcp.go:63] [bfd8adec45a196d0] [XXXXXXXXXXortRDP] tcp proxy listen port [8896] 2021/01/21 23:42:41 [I] [control.go:446] [bfd8adec45a196d0] new proxy [XXXXXXXXXXortRDP] success 2021/01/21 23:42:41 [I] [service.go:444] [b3c99ad587e6a41c] client login info: ip [192.168.1.159:59392] version [0.34.3] hostname [] os [windows] arch [amd64] 2021/01/21 23:42:41 [I] [tcp.go:63] [b3c99ad587e6a41c] [XXXXXXXXXHHHHHHH] tcp proxy listen port [20173] 2021/01/21 23:42:41 [I] [control.go:446] [b3c99ad587e6a41c] new proxy [XXXXXXXXXHHHHHHH] success 2021/01/21 23:42:41 [I] [tcp.go:63] [b3c99ad587e6a41c] [666666665.sbin_vm] tcp proxy listen port [8889] 2021/01/21 23:42:41 [I] [control.go:446] [b3c99ad587e6a41c] new proxy [666666665.sbin_vm] success 2021/01/21 23:42:41 [I] [tcp.go:63] [b3c99ad587e6a41c] [666666665.134server22] tcp proxy listen port [23422] 2021/01/21 23:42:41 [I] [control.go:446] [b3c99ad587e6a41c] new proxy [666666665.134server22] success 2021/01/21 23:42:41 [I] [tcp.go:63] [b3c99ad587e6a41c] [EEEEEEEEEE2XXXYYver] tcp proxy listen port [35000] 2021/01/21 23:42:41 [I] [control.go:446] [b3c99ad587e6a41c] new proxy [EEEEEEEEEE2XXXYYver] success 2021/01/21 23:42:41 [I] [tcp.go:63] [b3c99ad587e6a41c] [666666665.176server22] tcp proxy listen port [27622] 2021/01/21 23:42:41 [I] [control.go:446] [b3c99ad587e6a41c] new proxy [666666665.176server22] success 2021/01/21 23:42:41 [I] [tcp.go:63] [b3c99ad587e6a41c] [666666665.CIXXXYYver] tcp proxy listen port [8181] 2021/01/21 23:42:41 [I] [control.go:446] [b3c99ad587e6a41c] new proxy [666666665.CIXXXYYver] success 2021/01/21 23:42:41 [I] [tcp.go:63] [b3c99ad587e6a41c] [DDDDDDDDXXXYYver] tcp proxy listen port [10000] 2021/01/21 23:42:41 [I] [control.go:446] [b3c99ad587e6a41c] new proxy [DDDDDDDDXXXYYver] success 2021/01/21 23:42:41 [I] [tcp.go:63] [b3c99ad587e6a41c] [666666665.yongxing_pc] tcp proxy listen port [8890] 2021/01/21 23:42:41 [I] [control.go:446] [b3c99ad587e6a41c] new proxy [666666665.yongxing_pc] success 2021/01/21 23:42:41 [I] [tcp.go:63] [b3c99ad587e6a41c] [DDDDDD59server] tcp proxy listen port [8891] 2021/01/21 23:42:41 [I] [control.go:446] [b3c99ad587e6a41c] new proxy [DDDDDD59server] success 2021/01/21 23:42:41 [I] [tcp.go:63] [b3c99ad587e6a41c] [666666665.ConquestXXXYYver] tcp proxy listen port [5678] 2021/01/21 23:42:41 [I] [control.go:446] [b3c99ad587e6a41c] new proxy [666666665.ConquestXXXYYver] success 2021/01/21 23:42:41 [I] [tcp.go:63] [b3c99ad587e6a41c] [666666665.oneinstack173web443] tcp proxy listen port [20174] 2021/01/21 23:42:41 [I] [control.go:446] [b3c99ad587e6a41c] new proxy [666666665.oneinstack173web443] success 2021/01/21 23:42:41 [I] [tcp.go:63] [b3c99ad587e6a41c] [666666665.MSSQLTESTXXXYYver] tcp proxy listen port [20000] 2021/01/21 23:42:41 [I] [control.go:446] [b3c99ad587e6a41c] new proxy [666666665.MSSQLTESTXXXYYver] success 2021/01/21 23:42:41 [I] [tcp.go:63] [b3c99ad587e6a41c] [XXXXXXXXXXYYver] tcp proxy listen port [20002] 2021/01/21 23:42:41 [I] [control.go:446] [b3c99ad587e6a41c] new proxy [XXXXXXXXXXYYver] success 2021/01/21 23:42:42 [I] [proxy.go:103] [b3c99ad587e6a41c] [XXXXXXXXXXYYver] get a new work connection: [192.168.1.159:59392] 2021/01/21 23:42:42 [I] [proxy.go:103] [b3c99ad587e6a41c] [XXXXXXXXXXYYver] get a new work connection: [192.168.1.159:59392] 2021/01/21 23:42:42 [I] [proxy.go:103] [b3c99ad587e6a41c] [DDDDDDDDXXXYYver] get a new work connection: [192.168.1.159:59392] 2021/01/21 23:42:42 [I] [service.go:444] [5043827614628e93] client login info: ip [11.11.11.11:63531] version [0.34.1] hostname [] os [windows] arch [amd64] 2021/01/21 23:42:42 [I] [tcp.go:63] [5043827614628e93] [YYYYYYYYYYYYRDP] tcp proxy listen port [8897] 2021/01/21 23:42:42 [I] [control.go:446] [5043827614628e93] new proxy [YYYYYYYYYYYYRDP] success 2021/01/21 23:42:42 [I] [proxy.go:103] [b3c99ad587e6a41c] [DDDDDDDDXXXYYver] get a new work connection: [192.168.1.159:59392] 2021/01/21 23:42:42 [I] [proxy.go:103] [b3c99ad587e6a41c] [DDDDDDDDXXXYYver] get a new work connection: [192.168.1.159:59392] 2021/01/21 23:42:42 [I] [proxy.go:103] [b3c99ad587e6a41c] [XXXXXXXXXXYYver] get a new work connection: [192.168.1.159:59392] 2021/01/21 23:42:43 [I] [service.go:444] [eab1f2be0c69e775] client login info: ip [192.168.3.212:47400] version [0.34.3] hostname [] os [linux] arch [amd64] 2021/01/21 23:42:43 [I] [tcp.go:63] [eab1f2be0c69e775] [666666665.21777778085] tcp proxy listen port [8085] 2021/01/21 23:42:43 [I] [control.go:446] [eab1f2be0c69e775] new proxy [666666665.21777778085] success 2021/01/21 23:42:43 [I] [tcp.go:63] [eab1f2be0c69e775] [666666665.217777718080] tcp proxy listen port [18181] 2021/01/21 23:42:43 [I] [control.go:446] [eab1f2be0c69e775] new proxy [666666665.217777718080] success 2021/01/21 23:42:43 [I] [tcp.go:63] [eab1f2be0c69e775] [666666665.21777778020] tcp proxy listen port [21781] 2021/01/21 23:42:43 [I] [control.go:446] [eab1f2be0c69e775] new proxy [666666665.21777778020] success 2021/01/21 23:42:43 [I] [tcp.go:63] [eab1f2be0c69e775] [666666665.2177777RTMP] tcp proxy listen port [1935] 2021/01/21 23:42:43 [I] [control.go:446] [eab1f2be0c69e775] new proxy [666666665.2177777RTMP] success 2021/01/21 23:42:43 [I] [tcp.go:63] [eab1f2be0c69e775] [666666665.217777780] tcp proxy listen port [21780] 2021/01/21 23:42:43 [I] [control.go:446] [eab1f2be0c69e775] new proxy [666666665.217777780] success 2021/01/21 23:42:43 [I] [tcp.go:63] [eab1f2be0c69e775] [666666665.21777778888] tcp proxy listen port [8888] 2021/01/21 23:42:43 [I] [control.go:446] [eab1f2be0c69e775] new proxy [666666665.21777778888] success 2021/01/21 23:42:43 [I] [tcp.go:63] [eab1f2be0c69e775] [666666665.21777779000] tcp proxy listen port [9000] 2021/01/21 23:42:43 [I] [control.go:446] [eab1f2be0c69e775] new proxy [666666665.21777779000] success 2021/01/21 23:42:43 [I] [tcp.go:63] [eab1f2be0c69e775] [666666665.21777778084] tcp proxy listen port [8084] 2021/01/21 23:42:43 [I] [control.go:446] [eab1f2be0c69e775] new proxy [666666665.21777778084] success 2021/01/21 23:42:43 [I] [tcp.go:63] [eab1f2be0c69e775] [666666665.2177777122] tcp proxy listen port [21722] 2021/01/21 23:42:43 [I] [control.go:446] [eab1f2be0c69e775] new proxy [666666665.2177777122] success 2021/01/21 23:42:43 [I] [tcp.go:63] [eab1f2be0c69e775] [666666665.21777778083] tcp proxy listen port [8083] 2021/01/21 23:42:43 [I] [control.go:446] [eab1f2be0c69e775] new proxy [666666665.21777778083] success 2021/01/21 23:42:43 [I] [tcp.go:63] [eab1f2be0c69e775] [XXXXXXXXXUUUUUUUUUS] tcp proxy listen port [8443] 2021/01/21 23:42:43 [I] [control.go:446] [eab1f2be0c69e775] new proxy [XXXXXXXXXUUUUUUUUUS] success 2021/01/21 23:42:43 [I] [service.go:444] [11061fbdcabb6762] client login info: ip [192.168.3.212:42136] version [0.30.0] hostname [] os [linux] arch [arm] 2021/01/21 23:42:43 [I] [tcp.go:63] [11061fbdcabb6762] [XXXXY_YYY] tcp proxy listen port [8895] 2021/01/21 23:42:43 [I] [control.go:446] [11061fbdcabb6762] new proxy [XXXXY_YYY] success 2021/01/21 23:42:43 [I] [tcp.go:63] [11061fbdcabb6762] [XXXXDDWF] tcp proxy listen port [8892] 2021/01/21 23:42:43 [I] [control.go:446] [11061fbdcabb6762] new proxy [XXXXDDWF] success 2021/01/21 23:42:43 [I] [http.go:124] [11061fbdcabb6762] [XXXXXXXXX_ROUTER] http proxy listen for host [pc.jqk.cool] location [] group [] 2021/01/21 23:42:43 [I] [control.go:446] [11061fbdcabb6762] new proxy [XXXXXXXXX_ROUTER] success 2021/01/21 23:42:43 [I] [proxy.go:103] [5043827614628e93] [YYYYYYYYYYYYRDP] get a new work connection: [11.11.11.11:63531] 2021/01/21 23:42:43 [I] [proxy.go:103] [11061fbdcabb6762] [XXXXY_YYY] get a new work connection: [192.168.3.212:42136] 2021/01/21 23:42:45 [I] [proxy.go:103] [5043827614628e93] [YYYYYYYYYYYYRDP] get a new work connection: [11.11.11.11:63531] 2021/01/21 23:42:48 [I] [proxy.go:103] [b3c99ad587e6a41c] [XXXXXXXXXXYYver] get a new work connection: [192.168.1.159:59392] 2021/01/21 23:42:49 [I] [proxy.go:103] [5043827614628e93] [YYYYYYYYYYYYRDP] get a new work connection: [11.11.11.11:63531] 2021/01/21 23:42:49 [I] [proxy.go:103] [11061fbdcabb6762] [XXXXY_YYY] get a new work connection: [192.168.3.212:42136] 2021/01/21 23:42:49 [I] [proxy.go:103] [b3c99ad587e6a41c] [XXXXXXXXXXYYver] get a new work connection: [192.168.1.159:59392] 2021/01/21 23:42:49 [I] [proxy.go:103] [b3c99ad587e6a41c] [XXXXXXXXXXYYver] get a new work connection: [192.168.1.159:59392] 2021/01/21 23:42:49 [I] [proxy.go:103] [b3c99ad587e6a41c] [XXXXXXXXXXYYver] get a new work connection: [192.168.1.159:59392] 2021/01/21 23:42:49 [I] [proxy.go:103] [b3c99ad587e6a41c] [XXXXXXXXXXYYver] get a new work connection: [192.168.1.159:59392] 2021/01/21 23:42:49 [I] [proxy.go:103] [b3c99ad587e6a41c] [XXXXXXXXXXYYver] get a new work connection: [192.168.1.159:59392] 2021/01/21 23:42:49 [I] [proxy.go:103] [b3c99ad587e6a41c] [XXXXXXXXXXYYver] get a new work connection: [192.168.1.159:59392] 2021/01/21 23:42:49 [I] [proxy.go:103] [b3c99ad587e6a41c] [XXXXXXXXXXYYver] get a new work connection: [192.168.1.159:59392]

2021/01/21 23:50:28 [I] [proxy.go:103] [11061fbdcabb6762] [XXXXY_YYY] get a new work connection: [192.168.3.212:42136] `

Jan 22 '21 02:01 zsinba

就是在出现了一堆 get a new work connection 后那段时间出的的问题。我把日志脱敏了。

Jan 22 '21 02:01 zsinba

下面是FRPC的日志：从这个时间开始出错的。所以只取了这个时间的。

2021/01/21 23:42:39 [E] [control.go:158] [b3c99ad587e6a41c] work connection closed before response StartWorkConn message: EOF 2021/01/21 23:42:39 [E] [control.go:158] [b3c99ad587e6a41c] work connection closed before response StartWorkConn message: EOF 2021/01/21 23:42:39 [E] [control.go:158] [b3c99ad587e6a41c] work connection closed before response StartWorkConn message: EOF 2021/01/21 23:42:39 [E] [control.go:158] [b3c99ad587e6a41c] work connection closed before response StartWorkConn message: EOF 2021/01/21 23:42:39 [E] [control.go:158] [b3c99ad587e6a41c] work connection closed before response StartWorkConn message: EOF 2021/01/21 23:42:40 [W] [service.go:177] [b3c99ad587e6a41c] reconnect to server error: dial tcp 192.168.1.222:18080: connectex: No connection could be made because the target machine actively refused it. 2021/01/21 23:50:29 [E] [control.go:158] [b3c99ad587e6a41c] work connection closed before response StartWorkConn message: EOF 2021/01/21 23:50:29 [E] [control.go:158] [b3c99ad587e6a41c] work connection closed before response StartWorkConn message: EOF 2021/01/21 23:50:29 [E] [control.go:158] [b3c99ad587e6a41c] work connection closed before response StartWorkConn message: EOF 2021/01/21 23:50:29 [E] [control.go:158] [b3c99ad587e6a41c] work connection closed before response StartWorkConn message: EOF 2021/01/21 23:50:29 [E] [control.go:158] [b3c99ad587e6a41c] work connection closed before response StartWorkConn message: EOF 2021/01/21 23:50:30 [W] [service.go:177] [b3c99ad587e6a41c] reconnect to server error: dial tcp 192.168.1.222:18080: connectex: No connection could be made because the target machine actively refused it.

Jan 22 '21 03:01 zsinba

感觉受到了一波请求高峰（tcp 开始 listen 端口），然后导致了上述异常，“不应答”应该是最终现象。要具体排查问题原因，比如说瓶颈是什么，需要通过一些调试命令（可能偏系统层面）采集案发当时的信息。

我建议从系统端口占用/分配，比如说TIME_WAIT 也会占用端口数，以及服务本身的承载能力（qps）等方面入手排查。也建议做一些系统参数的优化，最好结合业务场景，看看网上有没有优化的示例。

Jan 22 '21 05:01 yuyulei

请求高峰，上限不会超过200。我是做了CDN过来的，有CDN服务器请求过来，高峰的时候，有20台服务器过来请求。从服务器性能来看，普通的一台4核20G的服务器跑的FRPS； FRPC是64核128G内存的服务器。主机性能应该不是原因，因为仅有一个端口出现问题，当这个端口出现问题的时候，其他端口的映射是正常的，没有任何问题。

所以，我觉得可能是FRPS和FRPC通信的时候，对某一个指定的端口，请求时返回等待，或者并发请求稍高（我说是200左右，并不是几千那么高），FRPC和FRPS之间就出问题了。

而且，这并不是必现的问题，两周左右出现一次。在出现问题的时候，我排查了FRPS和FRPC两个服务器，CPU、内存、IO、网络等情况，没有任何异常。都是很低的值的。

所以，是针对某个端口造成的死锁，这个死锁是来自于FRPS的。这个可能性最大。

Jan 22 '21 05:01 zsinba

从日志看当时 frpc 尝试连接 frps 的某个端口出现了问题，但是正如你所说，frpc 连接 frps的其他端口没问题。所以问题应该集中在出问题的端口上。但是，端口出问题的原因有很多，有可能是系统层面的原因。

“dial tcp 192.168.1.222:18080: connectex: No connection could be made because the target machine actively refused it.”

像这个错误看上去像是连接都没建立成功，换言之，很可能都没进入到 frps 代码逻辑或者直接被操作系统丢弃了。

我建议关注下当时网络层面的情况（重点端口，tcp 队列相关）。

Jan 27 '21 06:01 yuyulei

但是这个怎么解释，重启FRPS后，恢复正常了呢？我猜测原因肯定是与连接数有关。因为这个并不是马上出来，是运行一段时间后，积累了一些东西后才出来

与网络层情况相关的，这个不好跟踪了，使用的是一台群晖的NAS（1618+），常年不关机的。

Jan 27 '21 10:01 zsinba

我随便举个例子，tcp 握手的时候需要经过 sync queue，如果高并发时候这个 sync queue 过小加之后端处理不及时，就会出现tcp层面的阻塞（详见 sync flood 攻击）。虽然表象是说 frp 中有错误日志，但是并不能证明是 frp 引起的。哪怕 frp 重启就恢复了，这个例子一样能解释，因为 sync queue 清空了。

你还可以模拟压测，看看能否复现，可以分为实验组和对照组，一组是你现在使用的后端，一组是 mock 的后端。这样能排除后端服务带来的影响。

上述只是个人经验，并不一定适合你的场景，具体你自己可以网上寻找下别人是否有类似问题。

Jan 28 '21 02:01 yuyulei

我找到了必现的办法: 使用ab或者go-stress-testing进行压测. ab -n1000 -c1000 http://......这个地址(使用frp进行TCP映射的一个IIS地址)

压测几次,就必现了.这个HTTP服务器是正常的,但是通过FRP映射的端口就无法访问了．这时只能重启FRPS才能恢复．

Jan 29 '21 09:01 zsinba

直接压测IIS的时候，没有挂过，即使是短时间内不能服务，也只是在压测的时候，关闭压测就恢复了．而FRP无法恢复．当FRP不能访问时，IIS确认是正常的．所以这个肯定是和FRP有关,并不是系统的原因. 不过有可能是FRP的和系统交互的过程中有问题.

Jan 29 '21 09:01 zsinba

我把这个端口的frp代理,换成了Nginx来做, 目前没有问题了. 有十足的把握是frp的原因,同样的端口代理,使用Nginx没有这个问题. 使用FRP就会有. 对比测试是FRP的原因比较大. 再跑一个月看看nginx会不会出现同类问题.

Feb 02 '21 02:02 zsinba

感谢你提供的信息，我们这边也会压测分析下。

Feb 02 '21 02:02 yuyulei

有复现没

Feb 05 '21 05:02 zsinba

Reproduced on my side when the number of active connections on a port is over 1000 is there a way to solve this problem

Feb 19 '21 09:02 RazmikGevorgyan

Reproduced on my side when the number of active connections on a port is over 1000 is there a way to solve this problem Restart FRPS at regular intervals every night

Feb 19 '21 09:02 zsinba

@yuyulei 有进一步结论吗？

Feb 26 '21 01:02 zsinba

思路 -暴力美学：每 12 小时，重启所有的 docker 程序，彻底解决各种各样幺蛾子的问题；重启 docker 只有很短的时间，对用户影响很小； /usr/local/bin/docker restart $(docker ps -a | awk '{ print $1}' | tail -n +2)

同理： linux 系统，使用 crontab -e 增加

 0 6 * * * /usr/bin/docker restart $(docker ps -a | awk '{ print $1}' | tail -n +2)

也可以实现每天 6 点定时重启 docker 程序；

Jan 15 '22 03:01 ax2009live

思路 -暴力美学：每 12 小时，重启所有的 docker 程序，彻底解决各种各样幺蛾子的问题；重启 docker 只有很短的时间，对用户影响很小； /usr/local/bin/docker restart $(docker ps -a | awk '{ print $1}' | tail -n +2)

同理： linux 系统，使用 crontab -e 增加

0 6 * * * /usr/bin/docker restart $(docker ps -a | awk '{ print $1}' | tail -n +2) 也可以实现每天 6 点定时重启 docker 程序；

没有其他办法，就是这样做的。似乎没再出现这个问题。不确定是代码解决了，还是重启后解决了。

Jan 16 '22 08:01 zsinba

我使用TCP来传输本地机器的某一端口服务，使用时间/访问请求过多时，出现了服务器只接受拒绝响应的情况。端口由一开始的在通信，直接丢弃了对端口的监听。

Jun 05 '22 12:06 nymanyim

@fatedier we encountered this isse as well, reproduce steps described at here, do you have any plan to fix it? Thanks in advance.

Dec 09 '22 03:12 huxiaoliang

这个问题有进展了吗？

Dec 09 '22 04:12 zsinba

@fatedier we encountered this isse as well, reproduce steps described at here, do you have any plan to fix it? Thanks in advance.

Can not reproduce in my environment with the steps you described.

Dec 13 '22 15:12 fatedier

@fatedier we encountered this isse as well, reproduce steps described at here, do you have any plan to fix it? Thanks in advance.

Can not reproduce in my environment with the steps you described.

@leoryu could you please help me to clarify the stpes to reproduce this issue? Thanks.

Dec 14 '22 07:12 huxiaoliang

@huxiaoliang I cannot reproduce this issue in our env again recently. I will give the feedback when some new cases founded.

Jan 13 '23 01:01 leoryu

Issues go stale after 30d of inactivity. Stale issues rot after an additional 7d of inactivity and eventually close.

Aug 26 '23 00:08 github-actions[bot]