argo-workflows
argo-workflows copied to clipboard
feat: Add support for using ephemeral credentials for S3 artifacts. Fixes #5446 (#5446)
This PR addresses https://github.com/argoproj/argo-workflows/issues/5446
Motivation
We would like to use the new S3 access grants feature, which uses only temporary credentials. The aforementioned issue addresses temporary credentials in general (prior to recently though, this would have only been for IAM role).
Modifications
Firstly, some changes needed to be made in the Argo common pkg
repo: https://github.com/argoproj/pkg/pull/596
I am still waiting for these changes to be reviewed and therefore still have the forked dependency in my go.mod
. This is quite a small set of changes, as support for temporary credentials was already added to the AWS client in general previously.
Inside the Argo Workflows repo, I have made changes to how the S3 artifact driver is initialised, which mainly includes looking for the session token and using it if applicable. There is also a change on the workflow pod side to fetch the relevant secret if applicable.
Verification
I have written a couple unit tests to verify the driver and workflow creation behaviour. There was no test for creation of the driver, so I took the liberty of writing a small test to ensure the behaviour of specifically the changes I made. For the changes in s3.go
, the s3client
struct is not exported from argoproj/pkg
, so it made testing the changes I made impossible. I feel the changes made in this file are fairly small anyway.
I have also tested the changes by attempting to write into a S3 bucket in a workflow, using both IAM role and S3 access grant credentials. The behaviour is as expected; and when the credentials are invalid, or expired, an appropriate error message is returned.