aiobotocore icon indicating copy to clipboard operation
aiobotocore copied to clipboard

Unclosed coonnection

Open maretodoric opened this issue 7 months ago • 1 comments

Describe the bug

Unclosed connection
client_connection: Connection<ConnectionKey(host='bucket-name.s3.eu-central-1.amazonaws.com', port=443, is_ssl=True, ssl=None, proxy=None, proxy_auth=None, proxy_headers_hash=None)>

Warning shows up when using S3 client, get_object method.

This is the code I've used:

async def get_s3_object(bucket: str, filepath: str):
    filepath = str(filepath)
    cfg = load_settings()

    session = get_session()
    async with session.create_client('s3', region_name='eu-central-1',
                                     aws_secret_access_key=cfg.aws.secret_key,
                                     aws_access_key_id=cfg.aws.key) as client:
        # get object from s3
        try: s3_ob = await client.get_object(Bucket=bucket, Key=filepath)
        except Exception as e:
            if type(e).__name__ == 'NoSuchKey':
                raise FileNotFoundError(f"File '{basename(filepath)}' not found in bucket: '{bucket}'.")
            else:
                raise
        if s3_ob['ResponseMetadata']['HTTPStatusCode'] != 200:
            raise Exception(f"HTTP Status Code invalid. Response: {s3_ob}")
        else:
            return S3Object(bucket, filepath, s3_ob['ContentLength'], s3_ob['ContentType'])

Checklist

  • [x] I have reproduced in environment where pip check passes without errors
  • [x] I have provided pip freeze results
  • [x] I have provided sample code or detailed way to reproduce
  • [x] I have tried the same code in botocore to ensure this is an aiobotocore specific issue
  • [ ] I have tried similar code in aiohttp to ensure this is is an aiobotocore specific issue - i wouldn't know how to replicate this with aiohttp
  • [x] I have checked the latest and older versions of aiobotocore/aiohttp/python to see if this is a regression / injection

pip freeze results

aioboto3==12.4.0
aiobotocore==2.12.3
aiofiles==22.1.0
aiohttp==3.8.4
aioitertools==0.11.0
aiomysql==0.2.0
aiosignal==1.3.1
alembic==1.11.3
anyio==3.7.1
async-timeout==4.0.2
asyncssh==2.13.2
attrs==23.1.0
beautifulsoup4==4.12.2
boto3==1.34.69
botocore==1.34.69
certifi==2023.5.7
cffi==1.15.1
charset-normalizer==2.1.1
click==8.1.4
cryptography==39.0.0
dccalertmanager @ git+https://dcc-gitlab.addiko.com/mtodoric/pydccalertmanager.git@c88cd8af2415394621b10781d693ac2f52022bb8
dccjenkins @ git+https://dcc-gitlab.addiko.com/mtodoric/pydccjenkins.git@befd2d732e2f633e7965d11daa58af83003a7a3e
dccpassbolt @ git+https://dcc-gitlab.addiko.com/mtodoric/pydccpassbolt.git@ee8fd08e569ebc7a5a68e230801fa2c9763ea9b9
dccwebtool @ file:///home/mtodoric/dev-in-progress/pydccwebtool
dnspython==2.0.0
ecs-logging==2.2.0
elastic-apm==6.22.3
elasticsearch==7.17.6
elasticsearch-dsl==7.4.0
exceptiongroup==1.1.2
fastapi==0.90.0
frozenlist==1.3.3
gitdb==4.0.10
GitPython==3.1.32
greenlet==2.0.2
h11==0.14.0
httpcore==0.16.3
httpx==0.23.3
idna==3.4
iniconfig==2.0.0
ipwhois==1.2.0
Jinja2==3.1.2
jmespath==1.0.1
Mako==1.2.4
MarkupSafe==2.1.3
multidict==6.0.4
packaging==23.1
pluggy==1.2.0
pycparser==2.21
pydantic==1.10.9
PyMySQL==1.1.0
pytest==7.4.0
python-dateutil==2.8.2
python-dotenv==1.0.1
python-gnupg==0.5.1
python-multipart==0.0.5
PyYAML==6.0
requests==2.28.1
rfc3986==1.5.0
s3transfer==0.10.1
six==1.16.0
SkPy==0.10.5
smmap==5.0.0
sniffio==1.3.0
soupsieve==2.4.1
SQLAlchemy==2.0.15
starlette==0.22.0
tomli==2.0.1
typing_extensions==4.7.1
urllib3==1.26.16
uvicorn==0.20.0
websockets==10.4
wrapt==1.16.0
yarl==1.9.2

Environment:

  • Python Version: 3.10.12
  • OS name and version: Ubuntu 22.04

Additional context I've first noticed this when using aioboto3, however when i was told to try and replicate this with aiobotocore - i did and it can be replicated. Warning itself doesn't appear to cause any issues but potential high number of unclosed connections could pose an issue.

maretodoric avatar Jul 08 '24 07:07 maretodoric