TCP connection leaks

I am having fd leaks in mysql connections.

From the runtime inspection of GC, all Python objects are properly garbage-collected. However the actual TCP connections are still alive.

objgraphql.by_type('socket.socket') only lists the 6 connections:

"<socket.socket fd=742, family=2, type=1, proto=6, laddr=('10.0.0.1', 45160), raddr=('10.0.0.2', 3306)>",
"<socket.socket fd=752, family=2, type=1, proto=6, laddr=('10.0.0.1', 42152), raddr=('10.0.0.2', 3306)>",
"<socket.socket fd=753, family=2, type=1, proto=6, laddr=('10.0.0.1', 41736), raddr=('10.0.0.2', 3306)>",
"<socket.socket fd=762, family=2, type=1, proto=6, laddr=('10.0.0.1', 36052), raddr=('10.0.0.2', 3306)>",
"<socket.socket fd=737, family=2, type=1, proto=6, laddr=('10.0.0.1', 52222), raddr=('10.0.0.2', 3306)>",
"<socket.socket fd=778, family=2, type=1, proto=6, laddr=('10.0.0.1', 36620), raddr=('10.0.0.2', 3306)>"

but ss or lsof shows many more connections including the old connections:

...
python  2601911 root  715u  IPv4 305130877      0t0  TCP myserver:43446->dbserver:mysql (ESTABLISHED)
python  2601911 root  718u  IPv4 305132513      0t0  TCP myserver:44024->dbserver:mysql (ESTABLISHED)
python  2601911 root  722u  IPv4 305162297      0t0  TCP myserver:55118->dbserver:mysql (ESTABLISHED)
python  2601911 root  724u  IPv4 305159638      0t0  TCP myserver:52336->dbserver:mysql (ESTABLISHED)
python  2601911 root  729u  IPv4 305159665      0t0  TCP myserver:52374->dbserver:mysql (ESTABLISHED)
python  2601911 root  733u  IPv4 305162876      0t0  TCP myserver:53728->dbserver:mysql (ESTABLISHED)
python  2601911 root  737u  IPv4 305413976      0t0  TCP myserver:52222->dbserver:mysql (ESTABLISHED)
python  2601911 root  742u  IPv4 305169194      0t0  TCP myserver:45160->dbserver:mysql (ESTABLISHED)
python  2601911 root  752u  IPv4 305306885      0t0  TCP myserver:42152->dbserver:mysql (ESTABLISHED)
python  2601911 root  753u  IPv4 305305260      0t0  TCP myserver:41736->dbserver:mysql (ESTABLISHED)
python  2601911 root  762u  IPv4 305400262      0t0  TCP myserver:36052->dbserver:mysql (ESTABLISHED)
python  2601911 root  778u  IPv4 305679092      0t0  TCP myserver:36620->dbserver:mysql (ESTABLISHED)

note the last fd numbers (742, 752, 753, 762, 778) are the same.

I guess there is a remaining refcount to the underlying os socket. Should asyncmy call transport.shutdown() before transport.close() in Connection.close() ?

transport does not have the method shutdown. The problem here is probably you close event loop without cleanup. Underneath, transport.close() closes the underlying socket in the next event loop tick. See code, so you need to wait the rest to finish before closing the event loop.

You can try add a await asyncio.sleep(0) after calling conn.close() to see if it fixes the issue.