Add support to use IAM roles without credentials on s3 backup and restore

[slok]

Fixes #1682

This PR allows not using credentials and use instead node IAM roles or applications like Kiam or Kube2iam.

• Add new S3 client without credentials if not awsSecret present.
• The region when not using awsSecret will be inferred from the S3 endpoint.
• Add documentation and example of how to use based on IAM roles.
• Format some parts of the docs markdown.

I added Minio-go dependency only to infer the region, I thought it would be cleaner and had fewer corner cases/errors than implementing myself as it has a little bit of logic depending on the kind of regions/cases.

[etcd-bot]

Can one of the admins verify this patch?

[etcd-bot]

Can one of the admins verify this patch?

[etcd-bot]

Can one of the admins verify this patch?

[hexfusion]

@etcd-bot ok to test

[slok]

I didn't know that this app was used like (or affects as) a library so we mark the internal changes as breaking. Sorry about that. I'll add to the changelog!

[raoofm]

@hexfusion ping

[hexfusion]

@raoofm I think this PR needs some thought adding this as is would not be optimal. Let me try to get some cycles on this soon, thanks for the ping.

[danbeaulieu]

@hexfusion From my pov IAM role based access is much more typical than using keys and secrets. That said the auth config for the go sdk is well defined and IAM is documented as preferred https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html.

This code is already a bit odd with the requirement to create a shared config file as a secret and then write that out to the fs when it isn't one of the default credential providers for the go sdk.

Here is one proposal to help push this PR forward:

• Do the file management stuff automatically if the optional secret is set
• Create a custom credential chain that includes the shared config file, credential file and IAM provider. (I think this is needed since the shared config file is not in the default credential provider, alternatively if shared config is dropped then this isn't needed)
• Create the S3 client from the custom credential provider (similarly the default chain provider could be used if shared config is dropped)

Doing it this way delegates to the SDK and means a custom Auth object isn't needed?

[yzargari]

Does anyone have ETA on this? I need to setup backups and really don't want to create a user account with keys just to do that...

Thanks!

[jescarri]

Hey guys any progress on this?.

[Br3wm4ster]

Hey guys any help required here to get this merged?

I've been running this PR + Prometheus exporter one in prod for a month and it's been working w/o problems.

[slok]

@Br3wm4ster We have also been running this fork for the backups for several months (since the PR) and we didn't have any problems neither.

Apparently, this seems to be blocked/stopped because it wasn't optimal (although optimal is a matter of opinion, like perfection or software design, I guess) and we are waiting so the maintainers have some spare cycles to revisit this.

By the way, I can help fixing the things that are needed so we can have this feature on master or solving the conflicts with master at this moment.