AWSHTTPSConnectionPool - Pool is closed

[Itay4]

Describe the bug

running the following code in multithreading (tried to come up a minimal example of code to share), raises an exception inconsistently (once a day or so)

Expected Behavior

get_paginator response without errors

Current Behavior

An HTTP Client raised an unhandled exception: AWSHTTPSConnectionPool(host='HOSTNAME.s3.eu-central-1.amazonaws.com', port=443): Pool is closed.

Reproduction Steps

c = boto3.client("s3")
c.get_paginator("list_objects_v2").paginate(Bucket=bucket_name, MaxKeys=1)

Possible Solution

No response

Additional Information/Context

this might not be a bug, but bad usage - if you think thats the case, i would love to know what can be improved

SDK version used

1.24.12

Environment details (OS name and version, etc.)

Amazon EKS AMI Version 1.21.12

[tim-finnigan]

Hi @Itay4 thanks for reaching out. I wasn't able to reproduce the behavior you described. Could you provide your debug logs by adding boto3.set_stream_logger('') to your script? That could help give us more insight into what's going on. Please redact any sensitive information.

[Itay4]

from the debug logs:

AWSHTTPSConnectionPool(host='BUCKET-NAME.s3.amazonaws.com', port=443): Pool is closed.
File "/src/pkgs/botocore/httpsession.py", line 448, in send
    urllib_response = conn.urlopen(
  File "/src/pkgs/urllib3/connectionpool.py", line 692, in urlopen
    conn = self._get_conn(timeout=pool_timeout)
  File "/src/pkgs/urllib3/connectionpool.py", line 261, in _get_conn
    raise ClosedPoolError(self, "Pool is closed.")

might be related to https://github.com/urllib3/urllib3/issues/951

i hope that helps for investigating the issue, please let me know if there's any additional info i can provide

[tim-finnigan]

Hi @Itay4 thanks for your patience. Which version of urllib3 are you using? Have you tried installing the latest version of boto3 and running this?

[github-actions[bot]]

Greetings! It looks like this issue hasn’t been active in longer than five days. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.