docker-py icon indicating copy to clipboard operation
docker-py copied to clipboard

Published 20 hours ago • verified publisher icon docker

ResourceWarning: unclosed <socket.socket>

Open kelvich opened this issue 8 years ago • 6 comments

Hello.

It seems that in certain situations http socket isn't properly closed.

/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py:395: ResourceWarning: unclosed <socket.socket fd=7, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=/Users/stas/Library/Containers/com.docker.docker/Data/s60>

Python 3.5.1 on OSX.

Can be reproduced with following script:

import unittest
import docker

# Warning depends on code layout, so add here some ballast classes,
# probably that puts GC in specific position.

class A(object):
    def __init__(self, node):
        print('A')

class B(object):
    def __init__(self, node):
        print('B')

class C(object):
    def __init__(self, node):
        print('C')

class D(object):
    def __init__(self, node):
        print('D')

class E(object):
    def __init__(self, node):
        print('E')

class SingleNodePartition(object):

    def __init__(self, node):
        self.node = node
        self.docker_api = docker.Client()

    def container_exec(self, node, command):
        exec_id = self.docker_api.exec_create(node, command, user='root')
        output = self.docker_api.exec_start(exec_id)

    def start(self):
        self.container_exec(self.node, "ls")
        self.container_exec(self.node, "ls")
        self.container_exec(self.node, "ls")
        self.container_exec(self.node, "ls")

class RecoveryTest(unittest.TestCase):

    def test_node_partition(self):
        failure = SingleNodePartition('node3')
        failure.start()

if __name__ == '__main__':
    unittest.main()

kelvich avatar Nov 11 '16 12:11 kelvich

Still a problem in version 2.4.2 (observed on python:3.6.1-alpine)

hensing avatar Aug 17 '17 07:08 hensing

Have you tried calling docker_api.close()? See https://docker-py.readthedocs.io/en/stable/client.html#docker.client.DockerClient.close

Lekensteyn avatar Apr 24 '19 13:04 Lekensteyn

Even with that, I still get the occasional ResourceWarning error. Why doesn't UnixHTTPConnection have a close method? https://github.com/docker/docker-py/blob/master/docker/transport/unixconn.py Edit: it is not needed as it inherits from http.client.HTTPConnection which does implement close() that calls self.sock.close().

Minimal reproducer:

#!/usr/bin/env python3
import docker
import logging
logging.basicConfig(level=logging.DEBUG)

d = docker.from_env()
c = d.containers.run('alpine', auto_remove=True, detach=True, command='sleep 10')
res = c.exec_run('echo foo')
print(res)
c.kill()
d.close()

Debug patch:

--- a/usr/local/lib/python3.7/site-packages/docker/transport/unixconn.py
+++ b/usr/local/lib/python3.7/site-packages/docker/transport/unixconn.py
@@ -42,6 +42,15 @@
         sock.settimeout(self.timeout)
         sock.connect(self.unix_socket)
         self.sock = sock
+        print("Connected: %r" % self.sock)
+        if sock.fileno() in [6, 7]:
+            import traceback; traceback.print_stack()
+
+    def close(self):
+        print("closing: %r" % self.sock)
+        if self.sock.fileno() == 6:
+            import traceback; traceback.print_stack()
+        super().close()
 
     def putheader(self, header, *values):
         super(UnixHTTPConnection, self).putheader(header, *values)

Tested with Python 3.7.3 and docker-py 3.7.2 using: PYTHONTRACEMALLOC=25 python3 -Wd repro.py

Command output 
DEBUG:docker.utils.config:Trying paths: ['/Users/me/.docker/config.json', '/Users/me/.dockercfg']
DEBUG:docker.utils.config:Found file at path: /Users/me/.docker/config.json
DEBUG:docker.auth:Couldn't find auth-related section ; attempting to interpret as auth-only file
DEBUG:docker.auth:Config entry for key stackOrchestrator is not auth config
DEBUG:urllib3.connectionpool:http://localhost:None "POST /v1.35/containers/create HTTP/1.1" 201 90
DEBUG:urllib3.connectionpool:http://localhost:None "GET /v1.35/containers/6a13ffa029e9c9f9c65528d6f12db90b3c4d552d3cdbddae26985168c75ad038/json HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:http://localhost:None "POST /v1.35/containers/6a13ffa029e9c9f9c65528d6f12db90b3c4d552d3cdbddae26985168c75ad038/start HTTP/1.1" 204 0
  File "repro.py", line 8, in <module>
    res = c.exec_run('echo foo')
  File "/usr/local/lib/python3.7/site-packages/docker/models/containers.py", line 188, in exec_run
    workdir=workdir,
  File "/usr/local/lib/python3.7/site-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/docker/api/exec_api.py", line 79, in exec_create
    res = self._post_json(url, data=data)
  File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 289, in _post_json
    return self._post(url, data=json.dumps(data2), **kwargs)
  File "/usr/local/lib/python3.7/site-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 226, in _post
    return self.post(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 581, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1016, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 956, in send
    self.connect()
  File "/usr/local/lib/python3.7/site-packages/docker/transport/unixconn.py", line 47, in connect
    import traceback; traceback.print_stack()
DEBUG:urllib3.connectionpool:http://localhost:None "POST /v1.35/containers/6a13ffa029e9c9f9c65528d6f12db90b3c4d552d3cdbddae26985168c75ad038/exec HTTP/1.1" 201 74
  File "repro.py", line 8, in <module>
    res = c.exec_run('echo foo')
  File "/usr/local/lib/python3.7/site-packages/docker/models/containers.py", line 192, in exec_run
    demux=demux
  File "/usr/local/lib/python3.7/site-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/docker/api/exec_api.py", line 162, in exec_start
    stream=True
  File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 289, in _post_json
    return self._post(url, data=json.dumps(data2), **kwargs)
  File "/usr/local/lib/python3.7/site-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 226, in _post
    return self.post(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 581, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1016, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 956, in send
    self.connect()
  File "/usr/local/lib/python3.7/site-packages/docker/transport/unixconn.py", line 47, in connect
    import traceback; traceback.print_stack()
DEBUG:urllib3.connectionpool:http://localhost:None "POST /v1.35/exec/1b963d36b9bbcd24f54736e54250a316fed2f8ef7365be398e28e08f8a8cc541/start HTTP/1.1" 101 0
DEBUG:urllib3.connectionpool:http://localhost:None "GET /v1.35/exec/1b963d36b9bbcd24f54736e54250a316fed2f8ef7365be398e28e08f8a8cc541/json HTTP/1.1" 200 373
/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py:1200: ResourceWarning: unclosed <socket.socket fd=7, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=docker.sock>
  for i, one_value in enumerate(values):
Object allocated at (most recent call last):
  File "repro.py", lineno 8
    res = c.exec_run('echo foo')
  File "/usr/local/lib/python3.7/site-packages/docker/models/containers.py", lineno 192
    demux=demux
  File "/usr/local/lib/python3.7/site-packages/docker/utils/decorators.py", lineno 19
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/docker/api/exec_api.py", lineno 162
    stream=True
  File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", lineno 289
    return self._post(url, data=json.dumps(data2), **kwargs)
  File "/usr/local/lib/python3.7/site-packages/docker/utils/decorators.py", lineno 46
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", lineno 226
    return self.post(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", lineno 581
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", lineno 533
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", lineno 646
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", lineno 449
    timeout=timeout
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", lineno 600
    chunked=chunked)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", lineno 354
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", lineno 1229
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", lineno 1275
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", lineno 1224
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", lineno 1016
    self.send(msg)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", lineno 956
    self.connect()
  File "/usr/local/lib/python3.7/site-packages/docker/transport/unixconn.py", lineno 41
    sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
  File "repro.py", line 10, in <module>
    c.kill()
  File "/usr/local/lib/python3.7/site-packages/docker/models/containers.py", line 264, in kill
    return self.client.api.kill(self.id, signal=signal)
  File "/usr/local/lib/python3.7/site-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/docker/api/container.py", line 777, in kill
    res = self._post(url, params=params)
  File "/usr/local/lib/python3.7/site-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 226, in _post
    return self.post(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 581, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1016, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 956, in send
    self.connect()
  File "/usr/local/lib/python3.7/site-packages/docker/transport/unixconn.py", line 47, in connect
    import traceback; traceback.print_stack()
DEBUG:urllib3.connectionpool:http://localhost:None "POST /v1.35/containers/6a13ffa029e9c9f9c65528d6f12db90b3c4d552d3cdbddae26985168c75ad038/kill HTTP/1.1" 204 0
  File "repro.py", line 11, in <module>
    d.close()
  File "/usr/local/lib/python3.7/site-packages/docker/client.py", line 195, in close
    return self.api.close()
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 736, in close
    v.close()
  File "/usr/local/lib/python3.7/site-packages/docker/transport/basehttpadapter.py", line 8, in close
    self.pools.clear()
  File "/usr/local/lib/python3.7/site-packages/urllib3/_collections.py", line 95, in clear
    self.dispose_func(value)
  File "/usr/local/lib/python3.7/site-packages/docker/transport/unixconn.py", line 96, in <lambda>
    pool_connections, dispose_func=lambda p: p.close()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 420, in close
    conn.close()
  File "/usr/local/lib/python3.7/site-packages/docker/transport/unixconn.py", line 52, in close
    import traceback; traceback.print_stack()
Connected: <socket.socket fd=3, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=docker.sock>
Connected: <socket.socket fd=4, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=docker.sock>
Connected: <socket.socket fd=5, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=docker.sock>
Connected: <socket.socket fd=6, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=docker.sock>
Connected: <socket.socket fd=7, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=docker.sock>
Connected: <socket.socket fd=8, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=docker.sock>
ExecResult(exit_code=0, output=b'foo\n')
Connected: <socket.socket fd=7, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=docker.sock>
closing: <socket.socket fd=3, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=docker.sock>
closing: <socket.socket fd=4, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=docker.sock>
closing: <socket.socket fd=5, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=docker.sock>
closing: <socket.socket fd=6, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=docker.sock>
closing: <socket.socket fd=8, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=docker.sock>
closing: <socket.socket fd=7, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=docker.sock>

Lekensteyn avatar Apr 24 '19 14:04 Lekensteyn

Has this been fixed yet? I get this with docker py v4.2.0

caaespin avatar Feb 24 '20 22:02 caaespin

@caaespin presumably not, https://github.com/docker/docker-py/blob/master/docker/api/client.py has not been modified since March 15, 2019 while PR #2320 is still open.

Lekensteyn avatar Feb 26 '20 18:02 Lekensteyn

as a workaround:

def docker_from_env() -> ContextManager[docker.DockerClient]:
    return contextlib.closing(docker.from_env())

usage:

with docker_from_env() as client:
    ...

asottile-sentry avatar Sep 15 '22 19:09 asottile-sentry