OSGeo
AWS SSO has credential providers & token providers
What is the bug?
https://github.com/OSGeo/gdal/pull/11205 merged support for AWS SSO for vsis3 but the cached file can sometimes use sso_session_name for the hash instead of sso_start_url. This is because AWS SSO has a separate option which provides a clientID and clientSecret instead of an access token.
I didn't fully trace the code path but see the AWS Rust SDK where they refer to these 2 separate paths for the hash. The function that follows this parses the json file as well: https://github.com/awslabs/aws-sdk-rust/blob/d17757e4f975cd29622d3c6c00c7d917eb15e068/sdk/aws-config/src/sso/cache.rs#L138
Steps to reproduce the issue
Use AWS SSO
Versions and provenance
GDAL 3.10.2, released 2025/02/11
Additional context
No response
Following up on this bug because I'm running into the same issue using AWS SSO to connect to files in S3.
To reproduce the error:
docker pull ghcr.io/osgeo/gdal:ubuntu-small-3.10.2
docker run --rm -it ghcr.io/osgeo/gdal:ubuntu-small-3.10.2
curl -o awscli.zip https://awscli.amazonaws.com/awscli-exe-linux-"$(uname -i)".zip
unzip -q awscli.zip
./aws/install
Set up AWS SSO using:
aws configure sso
I then verify my connection to S3 with a public file path and try to connect to the file using gdalinfo:
root@1879990b86df:/# aws s3 ls sentinel-s1-rtc-indigo/tiles/RTC/1/IW/14/T/PN/2020/S1A_20200801_14TPN_ASC/ --profile MY_PROFILE
PRE S1A_IW_GRDH_1SDV_20200801T003023_20200801T003048_033706_03E810_267D/
2020-09-23 15:06:06 43757291 Gamma0_VH.tif
2020-09-23 15:06:00 43538532 Gamma0_VV.tif
2020-09-23 15:06:04 20182343 local_incident_angle.tif
root@1879990b86df:/# gdalinfo /vsis3/sentinel-s1-rtc-indigo/tiles/RTC/1/IW/14/T/PN/2020/S1A_20200801_14TPN_ASC/Gamma0_VV.tif
ERROR 15: AWS_SECRET_ACCESS_KEY and AWS_NO_SIGN_REQUEST configuration options not defined, and /root/.aws/credentials not filled
gdalinfo failed - unable to open '/vsis3/sentinel-s1-rtc-indigo/tiles/RTC/1/IW/14/T/PN/2020/S1A_20200801_14TPN_ASC/Gamma0_VV.tif'.
If I then set my default profile, export AWS_DEFAULT_PROFILE=MY_PROFILE, and run it all again, I get a new error.
root@1879990b86df:/# export AWS_DEFAULT_PROFILE=MY_PROFILE
root@1879990b86df:/# aws s3 ls sentinel-s1-rtc-indigo/tiles/RTC/1/IW/14/T/PN/2020/S1A_20200801_14TPN_ASC/Gamma0_VV.tif
2020-09-23 15:06:00 43538532 Gamma0_VV.tif
root@1879990b86df:/# gdalinfo /vsis3/sentinel-s1-rtc-indigo/tiles/RTC/1/IW/14/T/PN/2020/S1A_20200801_14TPN_ASC/Gamma0_VV.tif
ERROR 3: Cannot open file '/root/.aws/sso/cache/c4e055ec5328f8adc743da4443d3485dd0a39ca4.json'
ERROR 3: Load json file /root/.aws/sso/cache/c4e055ec5328f8adc743da4443d3485dd0a39ca4.json failed
ERROR 1: Cannot find file /root/.aws/sso/cache/c4e055ec5328f8adc743da4443d3485dd0a39ca4.json
ERROR 3: Cannot open file '/root/.aws/sso/cache/c4e055ec5328f8adc743da4443d3485dd0a39ca4.json'
ERROR 3: Load json file /root/.aws/sso/cache/c4e055ec5328f8adc743da4443d3485dd0a39ca4.json failed
ERROR 1: Cannot find file /root/.aws/sso/cache/c4e055ec5328f8adc743da4443d3485dd0a39ca4.json
ERROR 3: Cannot open file '/root/.aws/sso/cache/c4e055ec5328f8adc743da4443d3485dd0a39ca4.json'
ERROR 3: Load json file /root/.aws/sso/cache/c4e055ec5328f8adc743da4443d3485dd0a39ca4.json failed
ERROR 1: Cannot find file /root/.aws/sso/cache/c4e055ec5328f8adc743da4443d3485dd0a39ca4.json
ERROR 3: Cannot open file '/root/.aws/sso/cache/c4e055ec5328f8adc743da4443d3485dd0a39ca4.json'
ERROR 3: Load json file /root/.aws/sso/cache/c4e055ec5328f8adc743da4443d3485dd0a39ca4.json failed
ERROR 1: Cannot find file /root/.aws/sso/cache/c4e055ec5328f8adc743da4443d3485dd0a39ca4.json
ERROR 3: Cannot open file '/root/.aws/sso/cache/c4e055ec5328f8adc743da4443d3485dd0a39ca4.json'
ERROR 3: Load json file /root/.aws/sso/cache/c4e055ec5328f8adc743da4443d3485dd0a39ca4.json failed
ERROR 1: Cannot find file /root/.aws/sso/cache/c4e055ec5328f8adc743da4443d3485dd0a39ca4.json
ERROR 3: Cannot open file '/root/.aws/sso/cache/c4e055ec5328f8adc743da4443d3485dd0a39ca4.json'
ERROR 3: Load json file /root/.aws/sso/cache/c4e055ec5328f8adc743da4443d3485dd0a39ca4.json failed
ERROR 1: Cannot find file /root/.aws/sso/cache/c4e055ec5328f8adc743da4443d3485dd0a39ca4.json
gdalinfo failed - unable to open '/vsis3/sentinel-s1-rtc-indigo/tiles/RTC/1/IW/14/T/PN/2020/S1A_20200801_14TPN_ASC/Gamma0_VV.tif'.
I have files in ~/.aws/sso/cache/ but neither have the file name it's looking for. If I create a ~/.gdal/gdalrc file with my credentials as found in ~/.aws/cli/cache then it all seems to work.
~/.gdal/gdalrc example:
[credentials]
[.test]
path=/vsis3/
AWS_PROFILE=MY_PROFILE
AWS_ACCESS_KEY_ID=my_aws_access_key_id_from_cli_cache
AWS_SECRET_ACCESS_KEY=my_aws_secret_access_key_from_cli_cache
AWS_SESSION_TOKEN=my_aws_session_token_from_cli_cache
Then run gdalinfo /vsis3/sentinel-s1-rtc-indigo/tiles/RTC/1/IW/14/T/PN/2020/S1A_20200801_14TPN_ASC/Gamma0_VV.tif it finally connects:
Driver: GTiff/GeoTIFF
Files: /vsis3/sentinel-s1-rtc-indigo/tiles/RTC/1/IW/14/T/PN/2020/S1A_20200801_14TPN_ASC/Gamma0_VV.tif
Size is 5490, 5490
Coordinate System is:
PROJCRS["WGS 84 / UTM zone 14N",
BASEGEOGCRS["WGS 84",
...
It seems there's a bug in 3.10.2 with aws sso looking for the wrong file.
It would be great if AWS could actually publish a specification of how all this is supposed to work. Studying the guts of various SDKs around in various languages isn't fun...
I saw issues in both the Rust and Go SDKs where AWS also got this wrong themselves. So yes they really should!