Support source_profile in .aws/config for cross-account IAM roles with Kubernetes service account

[m2rt]

Expected behavior and actual behavior.

This is an extension to #4058 and #6074 which are about using IAM roles in EKS/Kubernetes. Another common pattern is to use aws resources cross-account. For example S3 has data in one account and mapserver runs in different account because different departments are responsible for them or accounts specify different environments. One way to configure this is to use "source_profile" cli configuration option which is partly supported by GDAL.

Flow how it should be (when .aws/config is configured like below):

1. Kubernetes service account gets role with AssumeRoleWithWebIdentity from main profile.
2. GDAL uses the token from first step to sts:AssumeRole from another account from source_profile
3. GDAL will use the tokens received from step 2 to get things from s3.

How it is: When running gdalinfo to vsis3: ERROR 15: Cannot retrieve credentials for source profile webidentityprovider

Steps to reproduce the problem.

For cross account resource access there is this guide.

Example of .aws/config which works with aws s3 commands.

[profile webidentityprovider]
role_arn = arn:aws:iam::ACCOUNT:role/ROLENAMESOURCE
web_identity_token_file = /var/run/secrets/eks.amazonaws.com/serviceaccount/token 

[profile default]
role_arn = arn:aws:iam::DESTINATIONACCOUNT:role/ROLENAMEDESTINATION
source_profile = webidentityprovider

Operating system

Ubuntu 20.04 running on a AWS EKS k8s 1.19 cluster

GDAL version and provenance

osgeo/gdal:ubuntu-small-3.5.1 docker image

[rouault]

Is my understanding correct that the osStsRootUrl + "/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=" + roleArn + "&WebIdentityToken=" + webIdentityToken HTTP request should use the value of role_arn from the source_profile (so here role_arn = arn:aws:iam::ACCOUNT:role/ROLENAMESOURCE ?), rather than the one of the default profile

[m2rt]

As much as I understand it from this. Then first its needed to authenticate with arn:aws:iam::ACCOUNT:role/ROLENAMESOURCE and get a token from it, which is how assumerolewithwebidentity is currently set up. And then with that token assumerole arn:aws:iam::DESTINATIONACCOUNT:role/ROLENAMEDESTINATION which result tokens will be used to access s3.

AWS has a C library for authentication parts. There is a test for a similar situation but instead of webidentity authentication the first authentication is done with access key/secret in that test.

And they handle it all with profile_base_provider.

[rouault]

perhaps @ashangit can help clarify ?

[ashangit]

My understanding is the same than @m2rt .

• First a call to AssumeRoleWithWebIdentity with the webidentityprovider arn and associated web identity token.
• From this call we retrieve the AccessKey/SecretKey/SessionToken.
• Second call to AssumeRole with the default arn and appropriate signV4 of the request based on previous AccessKey/SecretKey/SessionToken

One side note both AssumeRoleWithWebIdentity and AssumeRole provide temporary security credentials so it will require to do both call each time we will refresh the role credential in GetOrRefreshTemporaryCredentialsForRole

@m2rt will not make this issue work but #6074 is not available in gdal 3.5.1 but will be in 3.6. Also a potential workaround will be to use Bucket Policies (https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-policy-language-overview.html) which permits to authorize a role from another account to perform some S3 actions on the bucket

[rouault]

@ashangit Isn't the request of that ticket just a variation of what you've implemented when AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE configuration options are set from [profile webidentityprovider] ?

One side note both AssumeRoleWithWebIdentity and AssumeRole provide temporary security credentials so it will require to do both call each time we will refresh the role credential in GetOrRefreshTemporaryCredentialsForRole

As far as I understand, AssumeRoleWithWebIdentity is just called once currently with the WebIdentity method from configuration options. Is there any reason why it should be called each time we have to refresh when using values from the AWS configuration file ? Or shouldn't both cases behave the same regarding tihs ?

[ashangit]

The AssumeRoleWithWebIdentity is called here also in the RefreshCredentials function https://github.com/OSGeo/gdal/blob/26752efb88cf6d9042614340f48ce97fde5ef281/port/cpl_aws.cpp#L1792, So should be recalled, no?

[rouault]

So should be recalled, no?

ah indeed