dnscontrol icon indicating copy to clipboard operation
dnscontrol copied to clipboard

FR: Add the ability to limit concurrency (similar to a "max threads" limit)

Open fritterhoff opened this issue 11 months ago • 1 comments

Background

As discussed in #3294 we are stumbeling into some strange issues with to low rate limits. While we can mitigate these in our case by increasing the limits it is foreseeable that other service providers won't provide this option(s).

Idea

Either introduce a global limit of concurrency using a flag or provide an option to limit the concurrency per provider.

fritterhoff avatar Jan 08 '25 06:01 fritterhoff

Some resources:

  • https://github.com/nozzle/throttler
  • https://pkg.go.dev/github.com/carlmjohnson/monterey-jack/taskpool
  • https://github.com/jamesrr39/semaphore

tlimoncelli avatar Feb 05 '25 17:02 tlimoncelli

I don't have access to a test account for Porkbun. I would appreciate it if someone could test the change and report back.

Please test https://github.com/StackExchange/dnscontrol/pull/3764 (branch tlim/edge102_connlimit). It adds a flag called --cmax which limits concurrency. I recommend --cmax 5 then work upwards.

Instructions on testing a branch is here: https://docs.dnscontrol.org/developer-info/test-a-branch

tlimoncelli avatar Sep 09 '25 21:09 tlimoncelli