go-nsq
go-nsq copied to clipboard
nsqio
fix(consumer): remove old nsqd connections if addresses change
Right now, in the event that nsqd addresses change, the old connections stay around in the consumer's connections map. This simply cleans up old connections from that map.
Normally, connections are removed from Consumer.connections in Consumer.onConnClose(), which does various other cleanup tasks, to keep things consistent.
Are you having an issue where the tcp connection just hangs and is never detected to be closed?
@ploxiln when does Consumer.onConnClose get called? It looks like it only gets called when you close the consumer but what happens to connections that don't point to a valid IP in the event that upstream IPs change?
When the connection has some error, including detecting TCP close, it goes through a sequence:
https://github.com/nsqio/go-nsq/blob/master/conn.go#L640
that eventually calls consumerConnDelegate.OnClose():
https://github.com/nsqio/go-nsq/blob/master/conn.go#L725
That calls Consumer.onConnClose():
https://github.com/nsqio/go-nsq/blob/0e8d7a73ef5e81b6c3e2bdc4a9a5fb9afa44a36b/delegates.go#L121
In cases where a server really just switches ip addresses abruptly, there is no TCP FIN or RST that can be sent from the original IP to close the connection, so it'll have to time-out eventually, but operating systems may not enable TCP "keep-alives" by default. But if you do enable them, they should cause the zombie connection to be closed in a few minutes. https://github.com/golang/go/commit/5bd7e9c54f946eec95d32762e7e9e1222504bfc1 https://tldp.org/HOWTO/TCP-Keepalive-HOWTO/usingkeepalive.html