Adding support for using certbot-dns-nsone with linked zones

[dittnamn]

My operating system is (include version):

Gentoo, profile 23.0

I installed Certbot with (snap, OS package manager, pip, certbot-auto, etc):

Portage

I ran this command and it produced this output:

certbot certonly --dns-nsone --dns-nsone-credentials /etc/letsencrypt/nsone.conf -d example1.tld -d \*.example1.tld -d example2.tld -d \*.example2.tld

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator dns-nsone, Installer None
Requesting a certificate for example1.tld and 3 more domains
Performing the following challenges:
dns-01 challenge for example1.tld
dns-01 challenge for example1.tld
dns-01 challenge for example2.tld
dns-01 challenge for example2.tld
Cleaning up challenges
Error adding TXT record: 403 Client Error: Forbidden for url: https://api.nsone.net/v1/zones/example2.tld/_acme-challenge.example2.tld/TXT
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

Certbot's behavior differed from what I expected because:

In my example, the example2.tld zone is a linked zone that points to example1.tld's zone, to avoid having to add duplicate DNS records for the two domains.

Requesting a zone info for example2.tld from an NS1 API tool shows:

...
link: example1.tld
...

What I think should happen in these cases is that certbot-dns-nsone does a zone info request first, and if there's a link to another zone, the DNS records should be added to that zone instead. In cases where the linked zone is the same zone as another in the request, only one record should be added.

Here is a Certbot log showing the issue (if available):

I don't think the log file shows any more relevant parts for this, so I've skipped this. The 403 response is the relevant part.

Here is the relevant nginx server block or Apache virtualhost for the domain I am configuring:

This was with the certonly option.

[dittnamn]

I managed to track down part of the problem to Lexicon and created a patch to work with linked zones, see https://github.com/AnalogJ/lexicon/issues/1990

Certbot didn't really like this, however, as it will create different records on both domains and in this case these would be overwritten, as it's the same DNS zone:

certbot certonly --dns-nsone --dns-nsone-credentials /etc/letsencrypt/nsone.conf -d example1.tld -d \*.example1.tld -d example2.tld -d \*.example2.tld

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Requesting a certificate for example1.tld and 3 more domains
Waiting 30 seconds for DNS changes to propagate

Certbot failed to authenticate some domains (authenticator: dns-nsone). The Certificate Authority reported these problems:
  Domain: example2.tld
  Type:   unauthorized
  Detail: Incorrect TXT record "vwe...Po8" (and 1 more) found at _acme-challenge.example2.tld

  Domain: example2.tld
  Type:   unauthorized
  Detail: Incorrect TXT record "vwe...Po8" (and 1 more) found at _acme-challenge.example2.tld

Hint: The Certificate Authority failed to verify the DNS TXT records created by --dns-nsone. Ensure the above domains are hosted by this DNS provider, or try increasing --dns-nsone-propagation-seconds (currently 30 seconds).

What would the best approach to this be? I guess it wouldn't be permitted to use the same challenge data for more than one domain? Would it be possible to do it in a sequence instead of doing both domains at the same time?

[dittnamn]

Alright. What's left after changing the code in Lexicon turned out to be more general than just code for nsone. I made a simple patch that does work, but it's mostly made as a proof-of-concept that it works fine doing the verification in sequence. I'm not even sure if this is the best way forward and if added, this should be turned into an option to use (like --sequential).

When using linked domains, I had to change the code in acme that decides the challenge domain name because it otherwise would create a record called _acme-challenge.example2.tld.example1.tld. I'm not sure why the full domain is used in that request, but changing it like I did might break something else.

certbot-sequential-patch-testing.patch

[github-actions[bot]]

We've made a lot of changes to Certbot since this issue was opened. If you still have this issue with an up-to-date version of Certbot, can you please add a comment letting us know? This helps us to better see what issues are still affecting our users. If there is no activity in the next 30 days, this issue will be automatically closed.

[zoracon]

Right now we do not plan to have this in due to the increased complexity, but if there is more demand for it we can reconsider in the future.