docker-py
docker-py copied to clipboard
CancellableStream returned by docker_client.api.exec_start does not adequately release resources when closed, causing "I/O operation on closed file" errors
CancellableStream returned by docker_client.api.exec_start does not adequately release resources when closed, this leads to errors during python shutdown such as this:
File "/usr/lib/python3.13/site-packages/urllib3/response.py", line 1113, in close
self._fp.close()
File "/usr/lib/python3.13/http/client.py", line 432, in close
super().close() # set "closed" flag
File "/usr/lib/python3.13/http/client.py", line 445, in flush
self.fp.flush()
ValueError: I/O operation on closed file.
I believe this is related, if not the same issue, as: https://github.com/docker/docker-py/issues/3282
urllib3 Response objects need to be explicitly closed or they risk causing the problems above on exit.
(I will propose a patch shortly)
More context about this problem:
> pip freeze | grep docker && python --version && docker version
docker==7.1.0
Python 3.13.5
Client:
Version: 28.3.0
API version: 1.51
Go version: go1.24.4
Git commit: 38b7060a21
Built: Wed Jun 25 15:40:54 2025
OS/Arch: linux/amd64
Context: default
Server:
Engine:
Version: 28.3.0
API version: 1.51 (minimum version 1.24)
Go version: go1.24.4
Git commit: 265f709647
Built: Wed Jun 25 15:40:54 2025
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v2.1.3
GitCommit: c787fb98911740dd3ff2d0e45ce88cdf01410486.m
runc:
Version: 1.3.0
GitCommit:
docker-init:
Version: 0.19.0
GitCommit: de40ad0
The exception doesn't reproduce reliably. I've seen it happen on and off for months now, but I only now got what looked like a stable reproduction. Probably just lucky.
Slightly edited extract from our code that was triggering this:
exec_instance = docker_client.api.exec_create(
docker_container.id,
conf.command,
workdir=start_dir,
privileged=True,
tty=False,
environment=environment,
user='root' if conf.root else None
)
exec_output = docker_client.api.exec_start(
exec_instance['Id'],
tty=False,
stream=True,
demux=True,
)
try:
[..]
for (stdout_out, stderr_out) in exec_output:
[..]
finally:
exec_output.close()
# Workaround for https://github.com/docker/docker-py/issues/3345
exec_output._response.close()