Reload ProfileCredentialsProvider if ProfileFile changes on disk

[mina-asham]

In V2, the ProfileCredentialsProvider loads once and doesn't refresh/reload if the ProfileFile it was generated from got updated.

In V1, the ProfileCredentialsProvider has support for this try to reload every Duration X (customizable) if the profile file it was generated from have a newer last modified date.

This behaviour is also not documented in 1.11 to 2.0 Changelog, and due to that caused me a bug where I have a credentials file refreshed by a daemon in the background but my ProfileCredentialsProvider wasn't refreshing after migrating to V2.

Describe the Feature

Add the ability for ProfileCredentialsProvider to reload itself when the used ProfileFile gets changed on disk.

Is your Feature Request related to a problem?

Yes, this use case was covered by V1 SDK and wasn't documented in the changelog document, causing me to believe it works the same, upon migration I faced the issue and it took me a while to figure out what was going wrong.

Proposed Solution

I am proposing a couple of related changes:

• ProfileFile class, add support to:

1. Verify if the file was updated on disk since last loaded using last modified date
2. The ability to reload a new immutable ProfileFile from the current ProfileFile instance (basically re-builds it)

• ProfileCredentialsProvider class, on resolveCredentials calls, check the ProfileFile instance field to see if it changed, it it did reload the internal AwsCredentialsProvider instance field.

Additional Context

• [x] I may be able to implement this feature request (I have toyed with the proposed solution and it looks sensible)

This is related to another issue here: https://github.com/aws/aws-sdk-java-v2/issues/1327 (although that issue is mostly about refresh time but I believe the solution here would work for that issue as well)

[mina-asham]

I am going submit a PR to clarify the intent here, happy to completely modify it if we agree on some other solution!

[cenedhryn]

We apologize if that change was not apparent from the changelog; the log covered many features and it's possible this particular change in behavior was missed.

The main reason for not supporting this feature in V2 is, as far as I know, consistency with the other AWS SDK languages, which do not support it. We'll revisit this topic and come back with an answer as soon as possible.

[jeffzoch]

We were just bit by this too. Would love to see it implemented

[rgallagher-r7]

We have also been effected by this. I would see this as a potential blocker for migrating from v1 to v2 for anyone that relies on the v1 polling behaviour for cred refreshing. Is there a way around this?

[sstock]

As mentioned above and in boto/botocore#704 this is an issue when programs are forced to use short term credentials, in my case 60 minutes. Credentials that are supplied and updated via an external program that directly alters $HOME/.aws/credentials. Using the v2 SDK as-is means it is impossible to use/test programs that run beyond the credential's life time.

Currently the only solution is custom behavior that replaces the existing ProfileCredentialsProvider. Somewhat awkward when, as in my case, the remainder of DefaultCredentialsProvider needs to be left in place to support the various runtime environments.

I certainly feel that at least this should be clearly mentioned in the change log. It is true that when using fixed profile credentials this is likely not an issue. But with temporary credentials it represents a breaking change from the v1 SDK behavior.

That said, I believe the proposed solution is vastly preferable. When the program's code is available It avoids potentially replicating the functionality of DefaultCredentialsProvider with only a minor change. And it solves the issue for programs using ProfileCredentialsProvider (perhaps via DefaultCredentialsProvider) that cannot be altered.

[athrog]

Would like to express my support for this bugfix as well. At the very least, there should be an update to https://github.com/aws/aws-sdk-java-v2/blob/master/docs/LaunchChangelog.md that explicitly calls out the removal of this v1 feature.

[nobwyn]

I would like to support this bugfix as well - after spending hours on investigating why credentials aren't refreshed I was very surprised to find out, that such feature isn't available in SDK v2.

I think this feature is important in cases where dynamic credentials are expected (token rotation for example). If SDK supports dynamic credentials with InstanceProfileCredentialsProvider or ProcessProfileCredentialsProvider, I see no reason why it should not do the same with ProfileCredentialsProvider.

[DocX]

The #3487 update does not work with the "default" ProfileCredentialsProvider builder (and in turn DefaultCredentialsProvider chain that calls it). The builder uses fixedProfileFile supplier when no options are specified:

        ProfileCredentialsProvider credentials = ProfileCredentialsProvider.create();

which means that clients with no specific provider who fallback to DefaultCredentialsProvider does not reload the file:

        S3Client client = S3Client.builder()
                .region(Region.of("us-west-2"))
                .build();

The reloading however works when specifically setting the ProfileFileSupplier to ProfileFileSupplier.defaultSupplier():

        ProfileCredentialsProvider credentials = ProfileCredentialsProvider.builder()
                .profileFile(ProfileFileSupplier.defaultSupplier())
                .build();

        // or

        DefaultCredentialsProvider credentials = DefaultCredentialsProvider.builder()
                .profileFile(ProfileFileSupplier.defaultSupplier())
                .build();

In this case the provider reloads the credentials when the file changes, in both cases using directly the ProfileCredentialsProvider or DefaultCredentialsProvider (with the credential file in expected location).

SDK version: 2.25.25

---

Is there a reason why the "default" versions of the provider builder does not use the ProfileFileSupplier.defaultSupplier()?

[amalp12]

Would love to see this implemented as well.