gmqtt icon indicating copy to clipboard operation
gmqtt copied to clipboard

单节点启动,出现too many open files问题

Open mxyue opened this issue 3 years ago • 5 comments

客户端走websocket连接。

[yucuix@jgjapp-dev ~]$ sudo lsof -a -itcp -p 14723
COMMAND   PID USER   FD   TYPE    DEVICE SIZE/OFF NODE NAME
bin     14723 root    3u  IPv4 448738745      0t0  TCP localhost:ibm-mqisdp (LISTEN)
bin     14723 root    7u  IPv6 448739940      0t0  TCP *:secure-mqtt (LISTEN)
bin     14723 root    8u  IPv6 448739942      0t0  TCP *:us-cli (LISTEN)
bin     14723 root   10u  IPv4 448740762      0t0  TCP localhost:8084 (LISTEN)
bin     14723 root   11u  IPv4 448740763      0t0  TCP localhost:us-srv (LISTEN)
bin     14723 root   12u  IPv4 448740342      0t0  TCP localhost:56224->localhost:14003 (ESTABLISHED)
bin     14723 root   16u  IPv6 448782688      0t0  TCP localhost:secure-mqtt->localhost:50428 (ESTABLISHED)
bin     14723 root   20u  IPv6 448783484      0t0  TCP localhost:secure-mqtt->localhost:50324 (ESTABLISHED)
bin     14723 root   21u  IPv6 448781930      0t0  TCP localhost:secure-mqtt->localhost:50360 (ESTABLISHED)
bin     14723 root   30u  IPv6 448780932      0t0  TCP localhost:secure-mqtt->localhost:50126 (ESTABLISHED)
bin     14723 root   31u  IPv6 448778284      0t0  TCP localhost:secure-mqtt->localhost:49816 (CLOSE_WAIT)
bin     14723 root   32u  IPv6 448779220      0t0  TCP localhost:secure-mqtt->localhost:50232 (ESTABLISHED)
bin     14723 root   33u  IPv6 448782040      0t0  TCP localhost:secure-mqtt->localhost:50434 (ESTABLISHED)

端口连接看不出什么。开发环境下就几个连接,但是socket连接一直没释放断开的连接。

[root@jgjapp-dev fd]# ls /proc/31816/fd -l
总用量 0
lr-x------ 1 root root 64 8月   2 14:24 0 -> /dev/null
lrwx------ 1 root root 64 8月   2 14:24 1 -> socket:[448738742]
lrwx------ 1 root root 64 8月   2 14:24 10 -> socket:[448740762]
lrwx------ 1 root root 64 8月   2 14:24 11 -> socket:[448740763]
lrwx------ 1 root root 64 8月   2 14:24 12 -> socket:[448740342]
lrwx------ 1 root root 64 8月   2 14:24 13 -> socket:[448768168]
lrwx------ 1 root root 64 8月   2 14:24 14 -> socket:[448742405]
lrwx------ 1 root root 64 8月   2 14:24 15 -> socket:[448750807]
lrwx------ 1 root root 64 8月   2 14:24 16 -> socket:[448800654]
lrwx------ 1 root root 64 8月   2 14:24 17 -> socket:[448743237]
lrwx------ 1 root root 64 8月   2 14:24 18 -> socket:[448743240]
lrwx------ 1 root root 64 8月   2 14:24 19 -> socket:[448743301]
lrwx------ 1 root root 64 8月   2 14:24 2 -> socket:[448738742]
lrwx------ 1 root root 64 8月   2 14:24 20 -> socket:[448817234]
lrwx------ 1 root root 64 8月   2 14:24 21 -> socket:[448816826]
lrwx------ 1 root root 64 8月   2 14:24 22 -> socket:[448754806]
lrwx------ 1 root root 64 8月   2 14:24 23 -> socket:[448753159]
lrwx------ 1 root root 64 8月   2 14:24 24 -> socket:[448756526]
lrwx------ 1 root root 64 8月   2 14:24 25 -> socket:[448761346]
lrwx------ 1 root root 64 8月   2 14:24 26 -> socket:[448761448]
lrwx------ 1 root root 64 8月   2 14:24 27 -> socket:[448768765]
lrwx------ 1 root root 64 8月   2 14:24 28 -> socket:[448769909]
lrwx------ 1 root root 64 8月   2 14:24 29 -> socket:[448775662]
.........
lrwx------ 1 root root 64 8月   3 10:22 99 -> socket:[449476728]
lrwx------ 1 root root 64 8月   3 10:22 990 -> socket:[451674604]
lrwx------ 1 root root 64 8月   3 10:22 991 -> socket:[451679321]
lrwx------ 1 root root 64 8月   3 10:22 992 -> socket:[451676686]
lrwx------ 1 root root 64 8月   3 10:22 993 -> socket:[451682537]
lrwx------ 1 root root 64 8月   3 10:22 994 -> socket:[451680602]
lrwx------ 1 root root 64 8月   3 10:22 995 -> socket:[451682682]
lrwx------ 1 root root 64 8月   3 10:22 996 -> socket:[451684764]
lrwx------ 1 root root 64 8月   3 10:22 997 -> socket:[451695183]
lrwx------ 1 root root 64 8月   3 10:22 998 -> socket:[451686259]
lrwx------ 1 root root 64 8月   3 10:22 999 -> socket:[451686262]

mxyue avatar Aug 02 '22 06:08 mxyue

可能是有bug导致连接没有释放,可以观察一下监控数据看看有没有什么端倪。(要打开Prometheus插件,默认是开的)

curl 127.0.0.1:8082/metrics

DrmagicE avatar Aug 03 '22 14:08 DrmagicE

数量连接过多后,就没法创建新的连接了。打不开Prometheus连接 我后面再找时间看下。

mxyue avatar Aug 05 '22 08:08 mxyue

我用go 1.18去编译的。这个有没有影响呢

mxyue avatar Aug 09 '22 01:08 mxyue

comqtt support the distributed cluster.If you are interested, please refer to https://github.com/wind-c/comqtt. Sorry, gmqtt is also great.

wind-c avatar Sep 10 '22 14:09 wind-c

comqtt support the distributed cluster.If you are interested, please refer to https://github.com/wind-c/comqtt. Sorry, gmqtt is also great.

wind-c avatar Sep 10 '22 14:09 wind-c

@mxyue 有可能是鉴权失败,客户端不断开连接。导致连接开太多了。可以试一下按照这个comment先加一行能不能解决 https://github.com/DrmagicE/gmqtt/issues/177#issuecomment-1295829118

DrmagicE avatar Oct 29 '22 12:10 DrmagicE

更新代码后,内存泄漏的问题没有了。可能是那个问题造成的

mxyue avatar Feb 16 '23 05:02 mxyue