FR: Add the ability to limit concurrency (similar to a "max threads" limit)
Background
As discussed in #3294 we are stumbeling into some strange issues with to low rate limits. While we can mitigate these in our case by increasing the limits it is foreseeable that other service providers won't provide this option(s).
Idea
Either introduce a global limit of concurrency using a flag or provide an option to limit the concurrency per provider.
Some resources:
- https://github.com/nozzle/throttler
- https://pkg.go.dev/github.com/carlmjohnson/monterey-jack/taskpool
- https://github.com/jamesrr39/semaphore
I don't have access to a test account for Porkbun. I would appreciate it if someone could test the change and report back.
Please test https://github.com/StackExchange/dnscontrol/pull/3764 (branch tlim/edge102_connlimit). It adds a flag called --cmax which limits concurrency. I recommend --cmax 5 then work upwards.
Instructions on testing a branch is here: https://docs.dnscontrol.org/developer-info/test-a-branch