Auth credentials on AWS instance-based auth (ECS, EC2 metadata)

[awinder]

I believe there's a bit of non-standard behavior for instance-based auth versus other AWS clients, and I'm opening this as an issue in case it's purposeful. We ended up dropping paws for now and using lower-level aws modules because of it.

https://github.com/paws-r/paws/blob/main/paws.common/R/credential_providers.R#L98C1-L98C21

The AWS auth methods happen after a hard check on a ~/.aws/config file (or windows equivalent), but that isn't necessarily provided in EC2/ECS environments. You can look up the instance profile name from the IDMS endpoints and then pull temporary crews through it without any file-based config at all. The cloudyr aws packages all behave as expected with no file-based config, and python/java/nodejs apis I've all used firsthand without needing this config.

Wondering if this is a known issue, or something you'd look at changing / accept a PR on, since it seems divergent from normal behavior for API clients and maybe even the docs on pages like https://paws-r.github.io/developer_guide/credentials/#get-credentials-from-an-ec2-instance-or-container-role.

[DyfanJones]

I believe this is linked with https://github.com/paws-r/paws/issues/948 https://github.com/paws-r/paws/issues/441

Due to httr2 update the headers get lost. Please try the dev version in the mean time. I am planning to release paws.common shortly

install.packages('paws.common', repos = c('https://paws-r.r-universe.dev', 'https://cloud.r-project.org'))

[tyner]

Yesterday I encountered the same issue, and downgrading httr2 is indeed a viable short-term solution.

It would be good to document which version combinations (of paws.common and httr2) are problematic. For example we know paws.common version 0.8.4 in combination with httr2 version 1.2.1 is problematic. More generally, identifying a,b,c,x,y,z in something like the following could help users identify a suitable migration path (some users are stuck on R version < 4.1, so "just upgrade to the latest package versions" is not always a viable solution.)

if (utils::packageVersion("paws.common") < "a.b.c" && utils::packageVersion("httr2") >= "x.y.z") {
   stop("bad version combination -- either upgrade paws.common or downgrade httr2.")
}

[DyfanJones]

paws.common 0.8.7 has fixed this issue. It has been released onto the cran. So just waiting for cran build process to release to all platforms

[tyner]

Perhaps I encountered a different issue than the one fixed by version 0.8.7, but my issue seems to be resolved by version 0.8.6. Here are the combinations I tested, and their outcome:

httr2 version	paws.common version	outcome of paws.common::locate_credentials()
1.1.2	0.8.5	success
1.1.2	0.8.6	success
1.1.2	0.8.7	success
1.2.0	0.8.5	failure
1.2.0	0.8.6	success
1.2.0	0.8.7	success
1.2.1	0.8.5	failure
1.2.1	0.8.6	success
1.2.1	0.8.7	success

[DyfanJones]

Hi @tyner apologies for the confusion. paws.common 0.8.6 and 0.8.7 are basically the same. 0.8.7 just has some unit tests fixes giving time zoning issue (https://github.com/paws-r/paws/issues/955). I will update github releases shortly. I was just waiting for the cran automated testing to be completed before final release :)

[tyner]

No worries, makes total sense. As an aside, do you think the aws.signature and/or aws.ec2metadata packages might also not be compatible with the newer httr2 versions in this regard?

[DyfanJones]

It looks like they are using curl and jsonlite. So they should be fine and not affected by httr2.

https://github.com/cloudyr/aws.signature/blob/master/DESCRIPTION https://github.com/cloudyr/aws.ec2metadata/blob/master/DESCRIPTION