datadog-agent icon indicating copy to clipboard operation
datadog-agent copied to clipboard

Support for AWS SecretsManager in Secrets Helper

Open noseworthy opened this issue 6 months ago • 1 comments

What does this PR do?

This change extends the supported secrets providers to include AWS Secrets Manager. By providing an ARN to a secret as the id, the secrets helper can pull the secret from Secrets Manager provided that the agent was granted read access to the secret.

Unfortunately I'm not a go developer and haven't contributed to this repository before so I'm going to need some help to get this across the line.

Motivation

We're trying to configure DataDog Database Monitoring for our MongoDB Atlas cluster. All of our infrastructure is containerized in AWS running on Fargate. I couldn't see a way to launch an Agent to monitor Atlas in Fargate and provide the database credentials in a secure way. All of the documentation points to using docker labels which would necessitate providing the database credentials in plaintext in the task definition.

I saw that the agent has some support for pulling secrets from external services, but only from files or kubernetes secrets. There's mention of AWS Secrets Manager but we'd have to provide some sort of executable to provide the secrets. I figured it made sense to extend the existing functionality to include support for AWS Secrets Manager.

This issue was highlighted in https://github.com/DataDog/datadog-agent/issues/9676

Describe how you validated your changes

I've added a new unit test for the function to get secrets from AWS Secrets Manager.

Unfortunately I'm not sure how to effectively validate these changes, I'm very sorry. This PR is more of a proof of concept. I'm not a go developer and have little experience with this repository. I'm happy to hand this off to someone with more experience or work with someone to get the changes adopted though!

Possible Drawbacks / Trade-offs

Additional Notes

noseworthy avatar Jun 24 '25 21:06 noseworthy

CLA assistant check
All committers have signed the CLA.

bits-bot avatar Jun 24 '25 21:06 bits-bot

Hello @noseworthy,

Thank you for sending along this contribution for the Datadog Agent. Despite being new to golang your work here is quite good. Unfortunately, we're looking to move in a different direction for this functionality. Rather than expanding the Agent's own capabilities to access secrets directly from various cloud services, we've been planning on adding these capabilities to a separate binary, to match the modularity of the existing secret_backend_command feature. Much of this work has been happening in this repo, which we hope to soon package alongside the Agent releases. We do appreciate the work you put into this PR, thanks again.

dustmop avatar Jul 11 '25 20:07 dustmop

Thanks for taking a look, @dustmop!

Sounds good. I'll close this then. We managed to work around this anyway using environment variables in the form of `%%env_<VARIABLE_NAME>%%'. From the docs it sounded like it was kubernetes exclusive functionality, but it worked fine for our ECS containers running on Fargate.

noseworthy avatar Jul 12 '25 18:07 noseworthy