Role based EC2 access to S3

[denyszhak]

Feature Request

The key parameter is required to access S3 using S3StorageDriver https://github.com/apache/libcloud/blob/7b3f55a3ac2ec8423555cd9bac3a42697ffc502c/libcloud/storage/drivers/s3.py#L1224, even if provided it fails down in the code in the absence of secret here https://github.com/apache/libcloud/blob/trunk/libcloud/common/aws.py#L313

Can you suggest the usage of your client for role-based EC2 access to S3 or the possibility to add it? (where key and secret are now known well in advance but using STS for temporary access by tokens)

Thanks!

[denyszhak]

@Kami Does it make sense to add support for retrieving keys from instance metadata as a part of the library for every provider where it makes sense? I could review and propose an MR. If you don't want to have it in the library then let me know

Thanks!

[denyszhak]

@Kami Any chance you can provide your input here?

[jan-mue]

@denyszhak you can try this code to use the credentials from an IAM instance profile on EC2:

import boto3
from libcloud.storage.types import Provider
from libcloud.storage.providers import get_driver

session = boto3.Session()
credentials = session.get_credentials().get_frozen_credentials()
cls = get_driver(Provider.S3)
driver = cls(region="region", key=credentials.access_key, secret=credentials.secret_key, token=credentials.token)

If you have some long-running operations that use this libcloud driver, you might have to refresh the credentials, though.

[adityapatawari-uh]

Is there a fix for this problem? I know we can use boto3 to get the credentials, but at that point, I might as well use boto3 all the way and remove libcloud, which is not what I want to do. Any suggestions?