aws.s3
aws.s3 copied to clipboard
Do we have the ability to use assumed roles?
Hi,
We have a users that access s3 buckets in various accounts using one set of access keys by means of assumed roles.
The setup we use is pretty much that which is recommended by AWS and used by boto.
That is, using credentials and config files: ~/.aws/credentials containing the access keys
[default]
aws_access_key_id = XXXXX
aws_secret_access_key = XXXXXXX
~/.aws/config containing the roles
[profile production]
role_arn = <some role user can assume in "production" account>
source_profile = default
However it appears aws.s3 is not reading the contents of ~/.aws/config
> aws.signature::use_credentials(profile = "production")
> bucketlist()
Error in do.call("rbind.data.frame", r[["Buckets"]]) :
second argument must be a list
Does aws.s3 support roles in ~/.aws/config files or am I maybe calling it incorrectly? Thanks
Credentials files are supported by config files are not (those are boto-specific as far as I know). You should be able to assume a role using aws.iam::assume_role()
but you would need to supply the ARN directly to the function.
Happy to hear ideas for how to improve compatibility with other SDKs.
Thanks, you information helped me a lot. These are the steps I took: Add credentials to ~/.Renviron
AWS_ACCESS_KEY_ID = XXXXX
AWS_SECRET_ACCESS_KEY = XXXXXXXXXX
Start an R session and assume the r_myRole role with "use = TRUE" to cache the credentials
library("aws.s3")
aws.iam::assume_role(role = "arn:aws:iam::[SECOND_ACCOUNT_ID]:role/r_myRole", session = "mySession", use = TRUE)
Then I could list buckets in account "SECOND_ACCOUNT_ID"
getbucket("bucket-in-second-account")
However there's still one thing that doesn't work: I can not access buckets in a third account which have a bucket policy that allows access to users with SECOND_ACCOUNT_ID/r_myRole.
Using the aws cli it works using the same assumed role in second-account
cat ~/.aws/config
[profile second-account]
role_arn = arn:aws:iam::[SECOND_ACCOUNT_ID]:role/r_myRole
source_profile = default
# this works
aws s3 ls s3://bucket-in-third-account --profile second-account
# But the same operation doesn't work with aws.s3
getbucket("bucket-in-third-account")
Error in parse_aws_s3_response(r, Sig, verbose = verbose) :
Forbidden (HTTP 403)
I guess it's because it's a bucket policy and not an IAM policy that grants access here.
This is the bucket policy allowing the assumed role in second-account access to the bucket
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowAccessToSecondAccountUsers",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::[SECOND_ACCOUNT_ID]:role/r_myRole"
},
"Action": [
"s3:Get*",
"s3:List*"
],
"Resource": [
"arn:aws:s3:::bucket-in-third-account"
]
}
]
}
Thank for your time on this.
If I were to make one suggestion it would be to make the package work the same way as the aws cli for credentials. I think most data analysts probably use the aws cli for moving data to/from s3 so if the package could use the aws credentials files and profiles in the same way, it would make it easier for them and for platform admins who manage IAM roles.