external-dns icon indicating copy to clipboard operation
external-dns copied to clipboard

feat: update azure records concurrently

Open tanujd11 opened this issue 1 year ago • 8 comments

Description

DNS records concurrently for faster propagation of DNS records to Azure DNS Zone. This call need not be a blocking call.

Fixes https://github.com/kubernetes-sigs/external-dns/issues/4608

Checklist

  • [x] Unit tests updated
  • [ ] End user documentation updated

tanujd11 avatar Aug 08 '24 12:08 tanujd11

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please assign szuecs for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot avatar Aug 08 '24 12:08 k8s-ci-robot

Hi @tanujd11. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar Aug 08 '24 12:08 k8s-ci-robot

Thank you @tanujd11

deva26 avatar Aug 08 '24 13:08 deva26

Thanks for this interesting PR. Mmmm :thinking: this approach comes with many risks. When a user has 1000 endpoints, it will create 1000 requests in parallel.

Is this possible with Azure to send changes with batch ? Like it's done on AWS ?

mloiseleur avatar Aug 08 '24 13:08 mloiseleur

@mloiseleur , I understand it spins a goroutine per endpoint. Theoretically it is still safe to create 1000 or so requests in parallel. The actual thing which we will need to worry in this case is azure ratelimit which is about 200 Create or update calls. Also there is no batch request concept for Azure at the moment like AWS.

tanujd11 avatar Aug 08 '24 13:08 tanujd11

We can always limit the number of goroutines if that's what is needed for this PR to go through.

tanujd11 avatar Aug 08 '24 13:08 tanujd11

We can always limit the number of goroutines if that's what is needed for this PR to go through.

There are known user of external-dns with thousand of records. So yes, this PR needs to be improved to support them and avoid rate-limit.

mloiseleur avatar Aug 08 '24 14:08 mloiseleur

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar Oct 19 '24 08:10 k8s-ci-robot

I'm closing this PR, since there is no update since https://github.com/kubernetes-sigs/external-dns/pull/4665#issuecomment-2275968205

Feel free to re-open it or open a new one if needed. /close

mloiseleur avatar Dec 22 '24 17:12 mloiseleur

@mloiseleur: Closed this PR.

In response to this:

I'm closing this PR, since there is no update since https://github.com/kubernetes-sigs/external-dns/pull/4665#issuecomment-2275968205

Feel free to re-open it or open a new one if needed. /close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar Dec 22 '24 18:12 k8s-ci-robot