aiobotocore
aiobotocore copied to clipboard
Unclosed coonnection
Describe the bug
Unclosed connection
client_connection: Connection<ConnectionKey(host='bucket-name.s3.eu-central-1.amazonaws.com', port=443, is_ssl=True, ssl=None, proxy=None, proxy_auth=None, proxy_headers_hash=None)>
Warning shows up when using S3 client, get_object method.
This is the code I've used:
async def get_s3_object(bucket: str, filepath: str):
filepath = str(filepath)
cfg = load_settings()
session = get_session()
async with session.create_client('s3', region_name='eu-central-1',
aws_secret_access_key=cfg.aws.secret_key,
aws_access_key_id=cfg.aws.key) as client:
# get object from s3
try: s3_ob = await client.get_object(Bucket=bucket, Key=filepath)
except Exception as e:
if type(e).__name__ == 'NoSuchKey':
raise FileNotFoundError(f"File '{basename(filepath)}' not found in bucket: '{bucket}'.")
else:
raise
if s3_ob['ResponseMetadata']['HTTPStatusCode'] != 200:
raise Exception(f"HTTP Status Code invalid. Response: {s3_ob}")
else:
return S3Object(bucket, filepath, s3_ob['ContentLength'], s3_ob['ContentType'])
Checklist
- [x] I have reproduced in environment where
pip check
passes without errors - [x] I have provided
pip freeze
results - [x] I have provided sample code or detailed way to reproduce
- [x] I have tried the same code in botocore to ensure this is an aiobotocore specific issue
- [ ] I have tried similar code in aiohttp to ensure this is is an aiobotocore specific issue - i wouldn't know how to replicate this with aiohttp
- [x] I have checked the latest and older versions of aiobotocore/aiohttp/python to see if this is a regression / injection
pip freeze results
aioboto3==12.4.0
aiobotocore==2.12.3
aiofiles==22.1.0
aiohttp==3.8.4
aioitertools==0.11.0
aiomysql==0.2.0
aiosignal==1.3.1
alembic==1.11.3
anyio==3.7.1
async-timeout==4.0.2
asyncssh==2.13.2
attrs==23.1.0
beautifulsoup4==4.12.2
boto3==1.34.69
botocore==1.34.69
certifi==2023.5.7
cffi==1.15.1
charset-normalizer==2.1.1
click==8.1.4
cryptography==39.0.0
dccalertmanager @ git+https://dcc-gitlab.addiko.com/mtodoric/pydccalertmanager.git@c88cd8af2415394621b10781d693ac2f52022bb8
dccjenkins @ git+https://dcc-gitlab.addiko.com/mtodoric/pydccjenkins.git@befd2d732e2f633e7965d11daa58af83003a7a3e
dccpassbolt @ git+https://dcc-gitlab.addiko.com/mtodoric/pydccpassbolt.git@ee8fd08e569ebc7a5a68e230801fa2c9763ea9b9
dccwebtool @ file:///home/mtodoric/dev-in-progress/pydccwebtool
dnspython==2.0.0
ecs-logging==2.2.0
elastic-apm==6.22.3
elasticsearch==7.17.6
elasticsearch-dsl==7.4.0
exceptiongroup==1.1.2
fastapi==0.90.0
frozenlist==1.3.3
gitdb==4.0.10
GitPython==3.1.32
greenlet==2.0.2
h11==0.14.0
httpcore==0.16.3
httpx==0.23.3
idna==3.4
iniconfig==2.0.0
ipwhois==1.2.0
Jinja2==3.1.2
jmespath==1.0.1
Mako==1.2.4
MarkupSafe==2.1.3
multidict==6.0.4
packaging==23.1
pluggy==1.2.0
pycparser==2.21
pydantic==1.10.9
PyMySQL==1.1.0
pytest==7.4.0
python-dateutil==2.8.2
python-dotenv==1.0.1
python-gnupg==0.5.1
python-multipart==0.0.5
PyYAML==6.0
requests==2.28.1
rfc3986==1.5.0
s3transfer==0.10.1
six==1.16.0
SkPy==0.10.5
smmap==5.0.0
sniffio==1.3.0
soupsieve==2.4.1
SQLAlchemy==2.0.15
starlette==0.22.0
tomli==2.0.1
typing_extensions==4.7.1
urllib3==1.26.16
uvicorn==0.20.0
websockets==10.4
wrapt==1.16.0
yarl==1.9.2
Environment:
- Python Version: 3.10.12
- OS name and version: Ubuntu 22.04
Additional context I've first noticed this when using aioboto3, however when i was told to try and replicate this with aiobotocore - i did and it can be replicated. Warning itself doesn't appear to cause any issues but potential high number of unclosed connections could pose an issue.