feat(aws): use AWS profiles using .credentials file

[roehrijn]

trafficstars

Description

This PR introduces a new command line flag --aws-profile which can be used multiple times and allows to make use of AWS profiles in .credentials files (see also Configuration and credential file settings). Furthermore the PR allows to use multiple of these profiles at once in a way that it queries for available Route53 zones using credentials of each profile and later distinguishes between the profiles in order to make changes with proper credentials to the right Route53 zone.

We at Mercedes-Benz are running a fork with those changes now for several months in production.

Checklist

• [x] Unit tests updated
• [x] End user documentation updated

_{Jan Roehrich [email protected], Mercedes-Benz Tech Innovation GmbH, legal info/Impressum}

[k8s-ci-robot]

Welcome @roehrijn!

It looks like this is your first PR to kubernetes-sigs/external-dns 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/external-dns has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. :smiley:

[k8s-ci-robot]

Hi @roehrijn. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please assign johngmyers for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[tobiasgiese]

/ok-to-test

[johngmyers]

What is the value of having this complexity versus running an external-dns deployment per profile?

[johngmyers]

/retest

[roehrijn]

Hi @johngmyers, we run almost 1000 Kubernetes clusters which are concurrently accessing our Route53 zones through external-dns. To cope with Route53 throttling we need to distribute our zones over several AWS accounts which leads us to 6 profiles, currently. During some migration situations there might even be 9 profiles. Thus, the answer to your question about value has two dimensions. First, we have the complexity of running and managing up to 9 external-dns instances on up to 1000 clusters. On the other hand it is also about resource consumption. In the setup we use we need to request at least 60MB of memory per instance in order to work properly, in some situations a bit more. 60MB * 6 instances * 1000 clusters = a lot of additional EC2 costs.

[mloiseleur]

AWS profile is a feature of aws config file used in many aws compatible tools. Do you think you can rebase and finish this PR ?

[roehrijn]

Hi @mloiseleur, yes I'm going to rebase soon. Just had some other important topics on the table over the last view weeks.

[roehrijn]

Hi @mloiseleur, I rebased the PR. Do you see a chance to get this aspect merged? If yes, I would also take the effort to adapt the documentation accordingly.

[mloiseleur]

@roehrijn As soon as you finish and rebase this PR, I don't see why not ?

Note: It surely could help too to remove the WIP in the title.

[mloiseleur]

I have minor suggestion, and I think we are good to go.

[roehrijn]

@mloiseleur Thanks a lot for the quick response. Minor nits are fixed.

[roehrijn]

@mloiseleur @szuecs, please let me know if there is something left to do in order to get this merged. I would be very happy to avoid further rebases if possible, because it is a lot of work to validate function and valid documentation after each rebase. I'm eager to deal with missing aspects, however it would kindly ask to get to know them all together in order to avoid extra work.

[mloiseleur]

I understand. I think we are at the end. /lgtm /assign @szuecs for final review

[mloiseleur]

/retitle feat(aws): use AWS profiles using .credentials file

[szuecs]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: szuecs

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• ~~OWNERS~~ [szuecs]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[mloiseleur]

/lgtm