gdal icon indicating copy to clipboard operation
gdal copied to clipboard

Published 20 hours ago • verified publisher icon OSGeo

/vsis3/: support source_profile in .aws/config pointing to a profile …

Open rouault opened this issue 2 years ago • 15 comments

…with a web_identity_token_file (fixe #6320)

rouault avatar Sep 08 '22 18:09 rouault

@m2rt @ashangit I'd appreciate if you could review and/or test this. I only tested it through the unit tests I've added, not with a real setup.

rouault avatar Sep 08 '22 18:09 rouault

Tested it, did not work. S3 cli worked with the same config in the same container.

Built a docker image with: TARGET_IMAGE="test/gdal" ./docker/ubuntu-small/build.sh --proj master --release --gdal-repository rouault/gdal --gdal a424dd638b27e13c9ed47a18c7099cf7ba4ef40c

Had to monkey patch util.sh so I can see the build image in my docker images command: DOCKER_BUILDX_ARGS=("--platform" "${ARCH_PLATFORMS}" "--load")

Ran gdalinfo:

root@135b05e8d51d:/# gdalinfo /vsis3/MYBUCKET/myfile.jpg 
HTTP: Fetch(https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::MYACCOUNT:role/MYROLE&WebIdentityToken=MYTOKEN
)
HTTP: libcurl/7.81.0 GnuTLS/3.7.3 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.13

REPEAT THAT EXACT FETCH 5 times

ERROR 15: Cannot retrieve credentials for source profile webidentityprovider

Then I took the same fetch URL and ran curl with it. Got:

<AssumeRoleWithWebIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
  <AssumeRoleWithWebIdentityResult>
    <Audience>sts.amazonaws.com</Audience>
    <AssumedRoleUser>
      <AssumedRoleId>SOMEID:gdal</AssumedRoleId>
      <Arn>arn:aws:sts::MYACCOUNT:assumed-role/MYROLE/gdal</Arn>
    </AssumedRoleUser>
    <Provider>arn:aws:iam::MYACCOUNT:oidc-provider/oidc.eks.eu-central-1.amazonaws.com/id/OIDCID</Provider>
    <Credentials>
      <AccessKeyId>GENERATEDACCESSKEY</AccessKeyId>
      <SecretAccessKey>GENERATED SECRET</SecretAccessKey>
      <SessionToken>GENERATEDTOKEN</SessionToken>
      <Expiration>2022-09-14T12:52:13Z</Expiration>
    </Credentials>
    <SubjectFromWebIdentityToken>system:serviceaccount:mapserv:mapserver-s3-account-dev</SubjectFromWebIdentityToken>
  </AssumeRoleWithWebIdentityResult>
  <ResponseMetadata>
    <RequestId>MYREQUESTID</RequestId>
  </ResponseMetadata>
</AssumeRoleWithWebIdentityResponse>

How can I validate that I got the correct codebase?

root@135b05e8d51d:/# gdalinfo --version
GDAL 3.6.0dev, released 2022/99/99

m2rt avatar Sep 14 '22 12:09 m2rt

looking at how you built the Docker image, things should be fine. And the fact that there's a Action=AssumeRoleWithWebIdentity GET request shows that it uses that PR. So there's no Action=AssumeRole GET request after the AssumeRoleWithWebIdentity ? I can't really think of a way forward unless you build a debug version and single step starting at line 1550 of cpl_aws.cpp, or adding debug traces in GetConfigurationFromAssumeRoleWithWebIdentity() and GetConfigurationFromAWSConfigFiles(). Perhaps you could paste the whole (redacted where it makes sense) ~/.aws/config file

rouault avatar Sep 14 '22 12:09 rouault 

root@135b05e8d51d:/# cat ~/.aws/config
[profile webidentityprovider]
role_arn = arn:aws:iam::MYACCOUNT:role/MYROLE
web_identity_token_file = /root/.aws/token

[profile default]
role_arn = arn:aws:iam::MYOTHERACCOUNT:role/MYOTHERROLE
source_profile = webidentityprovider

If you tell me how to run this debug build and how to add debug traces, then why not. My only C experience is make install related :)

m2rt avatar Sep 14 '22 12:09 m2rt

I've pushed the fix_6320_debug in my rouault/gdal fork. You can build commit f13022631d06c6deeee1988adb8537b44b909fb0 and the run "gdalinfo /vsis3/MYBUCKET/myfile.jpg --debug on" and report the output

rouault avatar Sep 14 '22 21:09 rouault

Used find and replace over account names and other stuff, so it is consistent.

root@2b580559839a:/# cat ~/.aws/config 
[profile webidentityprovider]
role_arn = arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE
web_identity_token_file = /root/.aws/token

[profile default]
role_arn = arn:aws:iam::DESTACCOUNT:role/DESTROLE
source_profile = webidentityprovider



root@2b580559839a:/# gdalinfo /vsis3/BUCKET/FOLDER/FILE.jpg --debug on
AWS: GetConfigurationFromAWSConfigFiles() with profile = default
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::DESTACCOUNT:role/DESTROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = 
AWS: source_profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles() with profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = 
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = /root/.aws/token
AWS: GetConfigurationFromAWSConfigFiles() for source profile OK
AWS: GetConfigurationFromAssumeRoleWithWebIdentity start
AWS: GetConfigurationFromAssumeRoleWithWebIdentity refresh
AWS: GetConfigurationFromAssumeRoleWithWebIdentity osSTS_asuume_role_with_web_identity_URL=https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE&WebIdentityToken=TOKEN

HTTP: Fetch(https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE&WebIdentityToken=TOKEN
)
HTTP: libcurl/7.81.0 GnuTLS/3.7.3 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.13
AWS: AssumeRoleWithWebIdentity osAccessKeyId=

AWS: AssumeRoleWithWebIdentity osSecretAccessKey=

AWS: AssumeRoleWithWebIdentity osSessionToken=

AWS: AssumeRoleWithWebIdentity osExpiration=

AWS: GetConfigurationFromAWSConfigFiles() with profile = default
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::DESTACCOUNT:role/DESTROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = 
AWS: source_profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles() with profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = 
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = /root/.aws/token
AWS: GetConfigurationFromAWSConfigFiles() for source profile OK
AWS: GetConfigurationFromAssumeRoleWithWebIdentity start
AWS: GetConfigurationFromAssumeRoleWithWebIdentity refresh
AWS: GetConfigurationFromAssumeRoleWithWebIdentity osSTS_asuume_role_with_web_identity_URL=https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE&WebIdentityToken=TOKEN

HTTP: Fetch(https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE&WebIdentityToken=TOKEN
)
AWS: AssumeRoleWithWebIdentity osAccessKeyId=

AWS: AssumeRoleWithWebIdentity osSecretAccessKey=

AWS: AssumeRoleWithWebIdentity osSessionToken=

AWS: AssumeRoleWithWebIdentity osExpiration=

AWS: GetConfigurationFromAWSConfigFiles() with profile = default
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::DESTACCOUNT:role/DESTROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = 
AWS: source_profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles() with profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = 
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = /root/.aws/token
AWS: GetConfigurationFromAWSConfigFiles() for source profile OK
AWS: GetConfigurationFromAssumeRoleWithWebIdentity start
AWS: GetConfigurationFromAssumeRoleWithWebIdentity refresh
AWS: GetConfigurationFromAssumeRoleWithWebIdentity osSTS_asuume_role_with_web_identity_URL=https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE&WebIdentityToken=TOKEN

HTTP: Fetch(https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE&WebIdentityToken=TOKEN
)
AWS: AssumeRoleWithWebIdentity osAccessKeyId=

AWS: AssumeRoleWithWebIdentity osSecretAccessKey=

AWS: AssumeRoleWithWebIdentity osSessionToken=

AWS: AssumeRoleWithWebIdentity osExpiration=

AWS: GetConfigurationFromAWSConfigFiles() with profile = default
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::DESTACCOUNT:role/DESTROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = 
AWS: source_profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles() with profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = 
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = /root/.aws/token
AWS: GetConfigurationFromAWSConfigFiles() for source profile OK
AWS: GetConfigurationFromAssumeRoleWithWebIdentity start
AWS: GetConfigurationFromAssumeRoleWithWebIdentity refresh
AWS: GetConfigurationFromAssumeRoleWithWebIdentity osSTS_asuume_role_with_web_identity_URL=https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE&WebIdentityToken=TOKEN

HTTP: Fetch(https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE&WebIdentityToken=TOKEN
)
AWS: AssumeRoleWithWebIdentity osAccessKeyId=

AWS: AssumeRoleWithWebIdentity osSecretAccessKey=

AWS: AssumeRoleWithWebIdentity osSessionToken=

AWS: AssumeRoleWithWebIdentity osExpiration=

AWS: GetConfigurationFromAWSConfigFiles() with profile = default
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::DESTACCOUNT:role/DESTROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = 
AWS: source_profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles() with profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = 
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = /root/.aws/token
AWS: GetConfigurationFromAWSConfigFiles() for source profile OK
AWS: GetConfigurationFromAssumeRoleWithWebIdentity start
AWS: GetConfigurationFromAssumeRoleWithWebIdentity refresh
AWS: GetConfigurationFromAssumeRoleWithWebIdentity osSTS_asuume_role_with_web_identity_URL=https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE&WebIdentityToken=TOKEN

HTTP: Fetch(https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE&WebIdentityToken=TOKEN
)
AWS: AssumeRoleWithWebIdentity osAccessKeyId=

AWS: AssumeRoleWithWebIdentity osSecretAccessKey=

AWS: AssumeRoleWithWebIdentity osSessionToken=

AWS: AssumeRoleWithWebIdentity osExpiration=

AWS: GetConfigurationFromAWSConfigFiles() with profile = default
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::DESTACCOUNT:role/DESTROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = 
AWS: source_profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles() with profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = 
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = /root/.aws/token
AWS: GetConfigurationFromAWSConfigFiles() for source profile OK
AWS: GetConfigurationFromAssumeRoleWithWebIdentity start
AWS: GetConfigurationFromAssumeRoleWithWebIdentity refresh
AWS: GetConfigurationFromAssumeRoleWithWebIdentity osSTS_asuume_role_with_web_identity_URL=https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE&WebIdentityToken=TOKEN

HTTP: Fetch(https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE&WebIdentityToken=TOKEN
)
AWS: AssumeRoleWithWebIdentity osAccessKeyId=

AWS: AssumeRoleWithWebIdentity osSecretAccessKey=

AWS: AssumeRoleWithWebIdentity osSessionToken=

AWS: AssumeRoleWithWebIdentity osExpiration=

ERROR 15: Cannot retrieve credentials for source profile webidentityprovider
gdalinfo failed - unable to open '/vsis3/BUCKET/FOLDER/FILE.jpg'.

m2rt avatar Sep 15 '22 10:09 m2rt

ok, so it looks like the AssumeRoleWithWebIdentity request failed since the following

HTTP: Fetch(https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SOURCEACCOUNT:role/SOURCEROLE&WebIdentityToken=TOKEN
)
HTTP: libcurl/7.81.0 GnuTLS/3.7.3 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.13

returned empty credentials:

AWS: AssumeRoleWithWebIdentity osAccessKeyId=

Can you re-run with the CPL_CURL_VERBOSE config option set: gdalinfo /vsis3/MYBUCKET/myfile.jpg --debug on --config CPL_CURL_VERBOSE YES

rouault avatar Sep 15 '22 10:09 rouault 

root@2b580559839a:/# cat ~/.aws/config 
[profile webidentityprovider]
role_arn = arn:aws:iam::SRCACCOUNT:role/SRCROLE
web_identity_token_file = /root/.aws/token

[profile default]
role_arn = arn:aws:iam::DSTACCOUNT:role/DSTROLE
source_profile = webidentityprovider


root@2b580559839a:/# gdalinfo /vsis3/BUCKET/FOLDER/FILE.jpg --debug on --config CPL_CURL_VERBOSE YES
AWS: GetConfigurationFromAWSConfigFiles() with profile = default
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::DSTACCOUNT:role/DSTROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = 
AWS: source_profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles() with profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::SRCACCOUNT:role/SRCROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = 
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = /root/.aws/token
AWS: GetConfigurationFromAWSConfigFiles() for source profile OK
AWS: GetConfigurationFromAssumeRoleWithWebIdentity start
AWS: GetConfigurationFromAssumeRoleWithWebIdentity refresh
AWS: GetConfigurationFromAssumeRoleWithWebIdentity osSTS_asuume_role_with_web_identity_URL=https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SRCACCOUNT:role/SRCROLE&WebIdentityToken=TOKEN

HTTP: Fetch(https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SRCACCOUNT:role/SRCROLE&WebIdentityToken=TOKEN
)
HTTP: libcurl/7.81.0 GnuTLS/3.7.3 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.13
CURL_INFO_TEXT: Closing connection -1
AWS: AssumeRoleWithWebIdentity osAccessKeyId=

AWS: AssumeRoleWithWebIdentity osSecretAccessKey=

AWS: AssumeRoleWithWebIdentity osSessionToken=

AWS: AssumeRoleWithWebIdentity osExpiration=

AWS: GetConfigurationFromAWSConfigFiles() with profile = default
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::DSTACCOUNT:role/DSTROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = 
AWS: source_profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles() with profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::SRCACCOUNT:role/SRCROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = 
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = /root/.aws/token
AWS: GetConfigurationFromAWSConfigFiles() for source profile OK
AWS: GetConfigurationFromAssumeRoleWithWebIdentity start
AWS: GetConfigurationFromAssumeRoleWithWebIdentity refresh
AWS: GetConfigurationFromAssumeRoleWithWebIdentity osSTS_asuume_role_with_web_identity_URL=https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SRCACCOUNT:role/SRCROLE&WebIdentityToken=TOKEN

HTTP: Fetch(https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SRCACCOUNT:role/SRCROLE&WebIdentityToken=TOKEN
)
CURL_INFO_TEXT: Closing connection -1
AWS: AssumeRoleWithWebIdentity osAccessKeyId=

AWS: AssumeRoleWithWebIdentity osSecretAccessKey=

AWS: AssumeRoleWithWebIdentity osSessionToken=

AWS: AssumeRoleWithWebIdentity osExpiration=

AWS: GetConfigurationFromAWSConfigFiles() with profile = default
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::DSTACCOUNT:role/DSTROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = 
AWS: source_profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles() with profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::SRCACCOUNT:role/SRCROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = 
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = /root/.aws/token
AWS: GetConfigurationFromAWSConfigFiles() for source profile OK
AWS: GetConfigurationFromAssumeRoleWithWebIdentity start
AWS: GetConfigurationFromAssumeRoleWithWebIdentity refresh
AWS: GetConfigurationFromAssumeRoleWithWebIdentity osSTS_asuume_role_with_web_identity_URL=https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SRCACCOUNT:role/SRCROLE&WebIdentityToken=TOKEN

HTTP: Fetch(https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SRCACCOUNT:role/SRCROLE&WebIdentityToken=TOKEN
)
CURL_INFO_TEXT: Closing connection -1
AWS: AssumeRoleWithWebIdentity osAccessKeyId=

AWS: AssumeRoleWithWebIdentity osSecretAccessKey=

AWS: AssumeRoleWithWebIdentity osSessionToken=

AWS: AssumeRoleWithWebIdentity osExpiration=

AWS: GetConfigurationFromAWSConfigFiles() with profile = default
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::DSTACCOUNT:role/DSTROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = 
AWS: source_profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles() with profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::SRCACCOUNT:role/SRCROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = 
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = /root/.aws/token
AWS: GetConfigurationFromAWSConfigFiles() for source profile OK
AWS: GetConfigurationFromAssumeRoleWithWebIdentity start
AWS: GetConfigurationFromAssumeRoleWithWebIdentity refresh
AWS: GetConfigurationFromAssumeRoleWithWebIdentity osSTS_asuume_role_with_web_identity_URL=https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SRCACCOUNT:role/SRCROLE&WebIdentityToken=TOKEN

HTTP: Fetch(https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SRCACCOUNT:role/SRCROLE&WebIdentityToken=TOKEN
)
CURL_INFO_TEXT: Closing connection -1
AWS: AssumeRoleWithWebIdentity osAccessKeyId=

AWS: AssumeRoleWithWebIdentity osSecretAccessKey=

AWS: AssumeRoleWithWebIdentity osSessionToken=

AWS: AssumeRoleWithWebIdentity osExpiration=

AWS: GetConfigurationFromAWSConfigFiles() with profile = default
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::DSTACCOUNT:role/DSTROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = 
AWS: source_profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles() with profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::SRCACCOUNT:role/SRCROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = 
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = /root/.aws/token
AWS: GetConfigurationFromAWSConfigFiles() for source profile OK
AWS: GetConfigurationFromAssumeRoleWithWebIdentity start
AWS: GetConfigurationFromAssumeRoleWithWebIdentity refresh
AWS: GetConfigurationFromAssumeRoleWithWebIdentity osSTS_asuume_role_with_web_identity_URL=https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SRCACCOUNT:role/SRCROLE&WebIdentityToken=TOKEN

HTTP: Fetch(https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SRCACCOUNT:role/SRCROLE&WebIdentityToken=TOKEN
)
CURL_INFO_TEXT: Closing connection -1
AWS: AssumeRoleWithWebIdentity osAccessKeyId=

AWS: AssumeRoleWithWebIdentity osSecretAccessKey=

AWS: AssumeRoleWithWebIdentity osSessionToken=

AWS: AssumeRoleWithWebIdentity osExpiration=

AWS: GetConfigurationFromAWSConfigFiles() with profile = default
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::DSTACCOUNT:role/DSTROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = 
AWS: source_profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles() with profile = webidentityprovider
AWS: GetConfigurationFromAWSConfigFiles(): osAccessKeyId = 
AWS: GetConfigurationFromAWSConfigFiles(): osSecretAccessKey = 
AWS: GetConfigurationFromAWSConfigFiles(): osRoleArn = arn:aws:iam::SRCACCOUNT:role/SRCROLE
AWS: GetConfigurationFromAWSConfigFiles(): osSourceProfile = 
AWS: GetConfigurationFromAWSConfigFiles(): osWebIdentityTokenFile = /root/.aws/token
AWS: GetConfigurationFromAWSConfigFiles() for source profile OK
AWS: GetConfigurationFromAssumeRoleWithWebIdentity start
AWS: GetConfigurationFromAssumeRoleWithWebIdentity refresh
AWS: GetConfigurationFromAssumeRoleWithWebIdentity osSTS_asuume_role_with_web_identity_URL=https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SRCACCOUNT:role/SRCROLE&WebIdentityToken=TOKEN

HTTP: Fetch(https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SRCACCOUNT:role/SRCROLE&WebIdentityToken=TOKEN
)
CURL_INFO_TEXT: Closing connection -1
AWS: AssumeRoleWithWebIdentity osAccessKeyId=

AWS: AssumeRoleWithWebIdentity osSecretAccessKey=

AWS: AssumeRoleWithWebIdentity osSessionToken=

AWS: AssumeRoleWithWebIdentity osExpiration=

ERROR 15: Cannot retrieve credentials for source profile webidentityprovider
gdalinfo failed - unable to open '/vsis3/BUCKET/FOLDER/FILE.jpg'.


root@2b580559839a:/# curl "https://sts.us-east-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleSessionName=gdal&Version=2011-06-15&RoleArn=arn:aws:iam::SRCACCOUNT:role/SRCROLE&WebIdentityToken=TOKEN"
<AssumeRoleWithWebIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
  <AssumeRoleWithWebIdentityResult>
    <Audience>sts.amazonaws.com</Audience>
    <AssumedRoleUser>
      <AssumedRoleId>ASSUMEDROLEID:gdal</AssumedRoleId>
      <Arn>arn:aws:sts::SRCACCOUNT:assumed-role/SRCROLE/gdal</Arn>
    </AssumedRoleUser>
    <Provider>arn:aws:iam::SRCACCOUNT:oidc-provider/oidc.eks.eu-central-1.amazonaws.com/id/OIDCID</Provider>
    <Credentials>
      <AccessKeyId>ACCESSKEY</AccessKeyId>
      <SecretAccessKey>SECRETACCESSKEY</SecretAccessKey>
      <SessionToken>SESSIONTOKEN</SessionToken>
      <Expiration>2022-09-15T12:00:49Z</Expiration>
    </Credentials>
    <SubjectFromWebIdentityToken>system:serviceaccount:mapserv:mapserver-s3-account-dev</SubjectFromWebIdentityToken>
  </AssumeRoleWithWebIdentityResult>
  <ResponseMetadata>
    <RequestId>REQUESTID</RequestId>
  </ResponseMetadata>
</AssumeRoleWithWebIdentityResponse>

m2rt avatar Sep 15 '22 11:09 m2rt

"CURL_INFO_TEXT: Closing connection -1" is really intriguing I've pushed commit 90583a30a6d7e361239fe7cf4119b9d75c95239f with extra debug messages. Can you try with that gdalinfo /vsis3/MYBUCKET/myfile.jpg --debug on --config CPL_CURL_VERBOSE YES and possibly gdalinfo /vsis3/MYBUCKET/myfile.jpg --debug on --config CPL_CURL_VERBOSE YES --config CPL_HTTP_SKIP_SET_OPTIONS YES

rouault avatar Sep 15 '22 11:09 rouault

debuglogs.log

m2rt avatar Sep 16 '22 07:09 m2rt

ok so psResult->nStatus = 3 matches the following Curl effort: CURLE_URL_MALFORMAT, /* 3 */ So something is wrong in the encoding of the URL. Are there "special" (ie non alphanumeric) characters in the TOKEN passed to the WebIdentityToken query parameters ? Or maybe this is just the slash character in the RoleArn query parameter that must be URL encoded. I've just done that in commit b11d6bd41d2945a6f7ca8a4422a21f5ea737c1dd of the (non-debug) branch fix_6320

rouault avatar Sep 16 '22 08:09 rouault

Got the same: CURL_INFO_TEXT: Closing connection -1 Token has multiple things: -_. and aws doc even has % in token. https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRoleWithWebIdentity.html

m2rt avatar Sep 16 '22 10:09 m2rt

ah sorry, I actually messed up my cherry-picking from the debug branch to the regular one. So please try with commit a115adb56bfe4e569c92893859c48a21f8753607 But I'm not super confident this will work. I'm a bit puzzled why things work in curl command line, and not through API. Perhaps curl command line does some URL encoding behind the scene

rouault avatar Sep 16 '22 10:09 rouault

In the cli I wrapped the whole URL inbetween "" to make it work. Actually this worked as the first request succeeded but the next one failed. And it failed with curl also.

newerror.log

m2rt avatar Sep 16 '22 11:09 m2rt

ok, so there's a bit of progress as now the AssumeRoleWithWebIdentity works. I suspect the issue up to now was the trailing EOL character in the token file (that is now encoded as %0A, and presumably discarded by AWS servers) But now AssumeRole fails. I'm afraid I've no longer any clue on what's going on. Perhaps @ashangit has interest in helping/pursuing on this ?

rouault avatar Sep 16 '22 12:09 rouault

The issue might be my setup, will try to find time later this week to debug some more.

m2rt avatar Sep 27 '22 09:09 m2rt

Sorry for the delay was not able to take sometimes before

So I looked at the issue and I'm not sure why the query parameter mechanism to send credentials is not working on the AssumeRole action but I have also to admit that I have never use query parameter to send sign_v4 on AWS

I finally apply what I usually used for sign_v4 as indicated in AWS doc, meaning sending credentials in headers and it works.

Here is the ugly patch I applied to this PR to make it work: https://github.com/ashangit/gdal/commit/666874aa4c2c5cb15e572422ed2d3da36ab535e5

Not fully related to that PR but AssumeRole and AssumeRoleWithWebIdentity code are not using same pattern to define the sts url to query. It works but it is not consistent especially for that use case where we probably want to issue both call on the same sts endpoint relying on same configuration pattern

const CPLString stsRegionalEndpoints = VSIGetCredential(osPathForOption.c_str(),
                                                            "AWS_STS_REGIONAL_ENDPOINTS", "regional");

    std::string osStsDefaultUrl;
    if (stsRegionalEndpoints == "regional") {
        const CPLString osRegion = VSIGetCredential(osPathForOption.c_str(), "AWS_REGION", "us-east-1");
        osStsDefaultUrl = "https://sts." + osRegion + ".amazonaws.com";
    } else {
        osStsDefaultUrl = "https://sts.amazonaws.com";
    }

    const std::string osRegion(CPLGetConfigOption("AWS_STS_REGION", "us-east-1"));
    const std::string osHost(CPLGetConfigOption("AWS_STS_ENDPOINT", "sts.amazonaws.com"));

ashangit avatar Sep 29 '22 13:09 ashangit

@ashangit Thanks for the hint. I've tried to incorporate your suggestion of using header based authentication rather than through query parameters in commit 398fe487d56c29d19a09765f9b3c48529c68812b Could you / @m2rt test this ?

rouault avatar Oct 01 '22 17:10 rouault

FI, I've tested the last version of your branch and it works fine on my side.

ashangit avatar Oct 05 '22 07:10 ashangit