`BadStatusLine` errors from Docker API

[robin-wayve]

We get some infrequent/non-deterministic errors when using docker through testcontainers, a typical traceback looks like this (when trying to get logs from a container, but I think it has happened with other calls too):

  File "docker/docker/models/containers.py", line 306, in logs
    return self.client.api.logs(self.id, **kwargs)
  File "docker/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "docker/docker/api/container.py", line 869, in logs
    output = self._get_result(container, stream, res)
  File "docker/docker/api/client.py", line 460, in _get_result
    return self._get_result_tty(stream, res, self._check_is_tty(container))
  File "docker/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "docker/docker/api/client.py", line 456, in _check_is_tty
    cont = self.inspect_container(container)
  File "docker/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "docker/docker/api/container.py", line 772, in inspect_container
    self._get(self._url("/containers/{0}/json", container)), True
  File "docker/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "docker/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "requests/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "requests/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "requests/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "requests/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine('[...]'))

where the BadStatusLine contains what looks like the Docker Daemon API response body JSON, rather than HTTP/1.1 200 OK.

I'm not really sure how to investigate further as to where the fault might lie: I upgraded everything that seemed to be involved in the stack but it hasn't helped.

Versions

Ubuntu 20.04 Python 3.8.12

Pip

docker==5.0.3
requests==2.26.0
testcontainers==3.5.3
urllib3==1.26.9

Docker

Client: Docker Engine - Community
 Version:           20.10.14
 API version:       1.41
 Go version:        go1.16.15
 Git commit:        a224086
 Built:             Thu Mar 24 01:48:02 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.9
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.8
  Git commit:       79ea9d3
  Built:            Mon Oct  4 16:06:37 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.5.11
  GitCommit:        3df54a852345ae127d1fa3092b95168e4a88e2f8
 runc:
  Version:          1.0.3
  GitCommit:        v1.0.3-0-gf46b6ba
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0