s3 uploader does not use credentials stored in ~/.aws/credentials

[troyfontaine]

We're receiving the following error with the artifact upload functionality in the agent on macOS:

Failed to upload artifacts: Error creating uploader: Could not find a valid authentication strategy to connect to S3. Try setting BUILDKITE_S3_ACCESS_KEY and BUILDKITE_S3_SECRET_KEY

Reviewing the source code for the agent, it looks like the agent is dependent on the credentials being stored in environment variables and not checking if they're set in ~/.aws/credentials.

https://github.com/buildkite/agent/blob/0c2928974456f1ec8c753934779869a8d99dcb0b/agent/s3.go#L21-L43

The AWS SDK documentation states that you can pull in the values from the credentials file as follows:

sess, err := session.NewSession(&aws.Config{
    Region: aws.String("us-west-2")},
)

// Setup the S3 Upload Manager. Also see the SDK doc for the Upload Manager
// for more information on configuring part size, and concurrency.
//
// http://docs.aws.amazon.com/sdk-for-go/api/service/s3/s3manager/#NewUploader
uploader := s3manager.NewUploader(sess)

Perhaps a check to see if the default credentials file exists then use it for handling credentials, then fall back to the environment variables? Thanks!

[keithduncan]

We’re likely to get this for free when updating to aws-sdk-go-v2 in https://github.com/buildkite/agent/issues/1525 so I’ll put @eleanorakh’s in the assignees for this