aiohttp-s3-client icon indicating copy to clipboard operation
aiohttp-s3-client copied to clipboard

Failing to incomplete StaticCredentials() when one of ~/.aws/config and ~/.aws/credentials is absent

Open insan3d opened this issue 8 months ago • 1 comments

Problem Description

I'm trying to build a simple S3 caching web server using this library with aiohttp. My goal is to avoid the potential performance hit from using boto3 and only need basic S3 GET and PUT operations with a third-party S3 provider (not AWS).

The credentials initialization in credentials.py works well for mimicking boto3's configuration handling, but I'm facing an issue with the following logic:

try:
    credentials_paths_exists = (
        credentials_path.exists() and config_path.exists()
    )
except OSError:
    credentials_paths_exists = False

This code causes an issue because it expects both ~/.aws/credentials and ~/.aws/config to exist. However, in my case:

  1. I may not have a ~/.aws/credentials file, as I use URLCredentials() or environment variables for credentials, while configuration (e.g., region) is in ~/.aws/config.

  2. I may not have a ~/.aws/config file, since I specify the S3 endpoint directly in my app's configuration, and provide other settings (like the default region) via environment variables (I also use Docker secrets for ~/.aws/credentials).

This causes the following error to be raised: raise ValueError(f"Credentials {credentials!r} is incomplete")

Expected Behavior

I don’t believe both ~/.aws/credentials and ~/.aws/config should be required in all cases, especially for non-AWS S3 providers. In my case, I would like to bypass this check and rely solely on the initialization routine in the library, which works for interacting with third-party S3 providers.

Proposed Solution

It would be helpful if the library could handle cases where either ~/.aws/credentials or ~/.aws/config are missing, without raising an error. Alternatively, I could provide a configuration flag or a way to bypass this check.

Since rewriting the entire initialization logic in my app isn't ideal, I believe this is an issue worth addressing.

insan3d avatar Apr 05 '25 00:04 insan3d

Don't you want to create a PR?

mosquito avatar Apr 06 '25 09:04 mosquito