Secrets Manger Config not loading secrets

[amrutprabhu]

Type: Bug

Component: Secrets Manager

Describe the bug I am trying a simple example wherein I have my AWS credentials stored under an AWS profile called "personal"

Now I am trying to use this profile to fetch secrets from the secrets manager.

I have the following dependencies:

<dependencyManagement>
		<dependencies>
			<dependency>
				<groupId>io.awspring.cloud</groupId>
				<artifactId>spring-cloud-aws-dependencies</artifactId>
				<version>2.4.2</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>
			<dependency>
				<groupId>com.amazonaws</groupId>
				<artifactId>aws-java-sdk-bom</artifactId>
				<version>1.12.271</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>
		</dependencies>
	</dependencyManagement>
....
....
               <dependency>
  			<groupId>io.awspring.cloud</groupId>
			<artifactId>spring-cloud-starter-aws</artifactId>
		</dependency>
		<dependency>
			<groupId>io.awspring.cloud</groupId>
			<artifactId>spring-cloud-starter-aws-secrets-manager-config</artifactId>
		</dependency>

And in my application.yaml file

cloud:
  aws:
    credentials:
      profile-name: personal
    region:
      static: eu-central-1
   
spring:
  config:
    import: aws-secretsmanager:/secret/spring-boot-app 

The application fails because it can't fetch the secrets.

I did a little bit of digging, And I saw the DefaultAWSCredentialsProviderChain is getting built rather than the ProfileCredentialsProvider, according to this documentation here.

While debugging the application, I see the ProfileCredentialsProvider being one of the providers, Which is using the default provider because of the DefaultAWSCredentialsProviderChain initialization.

Now, I am not sure, Is my understanding of configuring the credentials is wrong? or is there something I am missing?

Also, I read this part here:

Spring Cloud AWS provides support for configuring an application context specific set of credentials that are used for each service call for requests done by Spring Cloud AWS components, except for the Parameter Store and Secrets Manager Configuration modules. Therefore, there must be exactly one configuration of the credentials for an entire application context.

from here

does it mean the autoconfiguration does not work with the spring-cloud-secrets-manager-config?

Thanks a lot for the help in advance.

[MatejNedic]

Hey @amrutprabhu tnx on reporting, Indeed this is a bug since CredentialsProvider is never taken into consideration when AWSSecretsManager client is being created.

This explains why DefaultAWSCredentialsProviderChain is used. ParameterStore is also affected by the same issue.

[maciejwalkowiak]

@MatejNedic correct me if i am wrong, but i think we've fixed in 3.0? In 2.x customizing bootstrap configuration can be done as explained in https://github.com/awspring/spring-cloud-aws/issues/205#issuecomment-1030964159

[MatejNedic]

Yes this is fixed in 3.0. Was thinking about adding same feature to 2.4.x or just modyfing docs since 3.0 is active now.

[amrutprabhu]

Maybe I can backport this change from 3.0. I'll try for first secrets manager then look into the parameter store. can you guide me through the change for 3.0 if you have any references for me to start with?

[MatejNedic]

Hey @amrutprabhu, Tnx for helping out!

If you check the 2.4.x branch, beans are being registered here.

As you can see CredentialsProperties are not not registered only AwsSecretsManagerProperties and AWSSecretsManager are.

Firstly we need to register them as well. In 3.0 you can see it here .

In 3.0 you will notice that when registering SecretsManagerClient AbstractAwsConfigDataLocationResolver.configure method is being called. You won't be able to do this in 2.4.x since Parameter Store and Secrets Manager are in different packages, but you can still use the method logic to configure AwsCredentialsProvider and later use it to configure AWSSecretsManager.

So in 2.4 you should apply same logic here.

If you get stuck feel free to reach out, I will try to respond as fast as possible.

[amrutprabhu]

Let me give see if I understand this right.

I see the CredentialsProperties is io.awspring.cloud.autoconfigure.context.properties.AwsCredentialsProperties in 2.4.x branch, which is present in the Spring-cloud-aws-autoconfigure module.

Now I have to use the properties from AwsCredentialsProperties to build the client in here via the AwsCredentialsProvider I guess using the implementation i.e ProfileCredentialsProvider. using the logic from configure here

But now, to create the AwsCredentialsProvider , I still cant use the class AwsCredentialsProperties inside here . since the class belongs to the autoconfigure module.

Am I missing something? or did I get it wrong ? 😅

[amrutprabhu]

Just my second thought about this change. I don't think we could create the provider in the Spring-cloud-aws-autoconfigure module and not may be the Spring-cloud-starter-aws-secret-manager. We then could fetch it from the bean context at this place here to create the client.

I am not sure how... Just an idea. 😄

[maciejwalkowiak]

@amrutprabhu apologies for late reply. @MatejNedic is on holidays and I did not pay attention to this issue as i try to offload everything Secrets Manager related to @MatejNedic :-)

We then could fetch it from the bean context at this place here to create the client.

This could work. There are quite a few "but"s - I think if we change the behavior of spring.config.import, we should also change the behavior of the legacy way of fetching secrets from Secrets Manager. Also, Parameter Store should follow the same pattern. The next thing is - while it's not technically breaking change as we break someone's build, it will change the runtime behavior.

To summarize - honestly I don't think we should implement it in 2.x. 3.x is just behind the corner, and spending more time on 2.x only postpones the 3.x release.

@amrutprabhu I appreciate your will to help and contribute and if it goes beyond this single issue I would be more than happy if you help us with other stuff for 3.x branch.

(btw, I have just realised that you are THIS Amrut 😊)

[maciejwalkowiak]

Closing since 3.0 is getting only closer.