certbot icon indicating copy to clipboard operation
certbot copied to clipboard

Published 20 hours ago • verified publisher icon certbot

Cloudflare limited scoped API tokens unable to find zone id (possible regression?)

Open ZetaTwo opened this issue 3 years ago • 7 comments

If you're having trouble using Certbot and aren't sure you've found a bug or request for a new feature, please first try asking for help at https://community.letsencrypt.org/. There is a much larger community there of people familiar with the project who will be able to more quickly answer your questions.

My operating system is (include version):

Ubuntu 20.04.2 LTS

I installed Certbot with (certbot-auto, OS package manager, pip, etc):

snap

I ran this command and it produced this output:

sudo certbot -v certonly --dns-cloudflare --dns-cloudflare-credentials /etc/cloudflare/secrets.ini -d "*.sub1.example.com" -d "sub2.example.com"

Error determining zone_id: 6003 Invalid request headers. Please confirm that you have supplied valid Cloudflare API credentials. (Did you copy your entire API token/key? To use Cloudflare tokens, you'll need the python package cloudflare>=2.3.1. This certbot is running cloudflare 2.3.1)

Certbot's behavior differed from what I expected because:

This seems to be a regression of the issue discussed in #7893. I have tried API keys with the following variants of permissions:

  • Zone.DNS:Edit (limited to example.com)
  • Zone.DNS:Edit, Zone.Zone:Read (limited to example.com)
  • Zone.DNS:Edit, Zone.Zone:Edit (limited to example.com)
  • Zone.DNS:Edit, Zone.Zone:Edit (with all zones allowed)

And I still get the same error every time.

Here is a Certbot log showing the issue (if available):

Logs are stored in /var/log/letsencrypt by default. Feel free to redact domains, e-mail and IP addresses as you see fit.
2021-03-23 23:59:11,495:DEBUG:acme.client:Storing nonce: 01041GoUuFXUC-GnyoKJWr3tWbHO0uSUufptPf-MvA4tgBo
2021-03-23 23:59:11,496:INFO:certbot._internal.auth_handler:Performing the following challenges:
2021-03-23 23:59:11,496:INFO:certbot._internal.auth_handler:dns-01 challenge for sub1.example.com
2021-03-23 23:59:11,497:INFO:certbot._internal.auth_handler:dns-01 challenge for sub2.example.com
2021-03-23 23:59:11,502:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.cloudflare.com:443
2021-03-23 23:59:12,430:DEBUG:urllib3.connectionpool:https://api.cloudflare.com:443 "GET /client/v4/zones?name=sub1.example.com&per_page=1 HTTP/1.1" 400 None
2021-03-23 23:59:12,435:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/snap/certbot-dns-cloudflare/current/lib/python3.8/site-packages/certbot_dns_cloudflare/_internal/dns_cloudflare.py", line 184, in _find_zone_id
    zones = self.cf.zones.get(params=params)  # zones | pylint: disable=no-member
  File "/snap/certbot-dns-cloudflare/current/lib/python3.8/site-packages/CloudFlare/cloudflare.py", line 622, in get
    return self._base.call_with_auth('GET', self._parts,
  File "/snap/certbot-dns-cloudflare/current/lib/python3.8/site-packages/CloudFlare/cloudflare.py", line 80, in call_with_auth
    return self._call(method, headers, parts,
  File "/snap/certbot-dns-cloudflare/current/lib/python3.8/site-packages/CloudFlare/cloudflare.py", line 473, in _call
    raise CloudFlareAPIError(code, message, error_chain)
CloudFlare.exceptions.CloudFlareAPIError: Invalid request headers

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/snap/certbot/1042/lib/python3.8/site-packages/certbot/_internal/auth_handler.py", line 70, in handle_authorizations
    resps = self.auth.perform(achalls)
  File "/snap/certbot/1042/lib/python3.8/site-packages/certbot/plugins/dns_common.py", line 57, in perform
    self._perform(domain, validation_domain_name, validation)
  File "/snap/certbot-dns-cloudflare/current/lib/python3.8/site-packages/certbot_dns_cloudflare/_internal/dns_cloudflare.py", line 77, in _perform
    self._get_cloudflare_client().add_txt_record(domain, validation_name, validation, self.ttl)
  File "/snap/certbot-dns-cloudflare/current/lib/python3.8/site-packages/certbot_dns_cloudflare/_internal/dns_cloudflare.py", line 107, in add_txt_record
    zone_id = self._find_zone_id(domain)
  File "/snap/certbot-dns-cloudflare/current/lib/python3.8/site-packages/certbot_dns_cloudflare/_internal/dns_cloudflare.py", line 201, in _find_zone_id
    raise errors.PluginError('Error determining zone_id: {0} {1}. Please confirm '
certbot.errors.PluginError: Error determining zone_id: 6003 Invalid request headers. Please confirm that you have supplied valid Cloudflare API credentials. (Did you copy your entire API token/key? To use Cloudflare tokens, you'll need the python package cloudflare>=2.3.1. This certbot is running cloudflare 2.3.1)

2021-03-23 23:59:12,438:DEBUG:certbot._internal.error_handler:Calling registered functions
2021-03-23 23:59:12,438:INFO:certbot._internal.auth_handler:Cleaning up challenges
2021-03-23 23:59:12,443:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.cloudflare.com:443
2021-03-23 23:59:13,305:DEBUG:urllib3.connectionpool:https://api.cloudflare.com:443 "GET /client/v4/zones?name=sub1.example.com&per_page=1 HTTP/1.1" 400 None
2021-03-23 23:59:13,310:DEBUG:certbot_dns_cloudflare._internal.dns_cloudflare:Encountered error finding zone_id during deletion: Error determining zone_id: 6003 Invalid request headers. Please confirm that you have supplied valid Cloudflare API credentials. (Did you copy your entire API token/key? To use Cloudflare tokens, you'll need the python package cloudflare>=2.3.1. This certbot is running cloudflare 2.3.1)
2021-03-23 23:59:13,318:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.cloudflare.com:443
2021-03-23 23:59:14,132:DEBUG:urllib3.connectionpool:https://api.cloudflare.com:443 "GET /client/v4/zones?name=sub2.example.com&per_page=1 HTTP/1.1" 400 None
2021-03-23 23:59:14,136:DEBUG:certbot_dns_cloudflare._internal.dns_cloudflare:Encountered error finding zone_id during deletion: Error determining zone_id: 6003 Invalid request headers. Please confirm that you have supplied valid Cloudflare API credentials. (Did you copy your entire API token/key? To use Cloudflare tokens, you'll need the python package cloudflare>=2.3.1. This certbot is running cloudflare 2.3.1)
2021-03-23 23:59:14,139:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/snap/certbot-dns-cloudflare/current/lib/python3.8/site-packages/certbot_dns_cloudflare/_internal/dns_cloudflare.py", line 184, in _find_zone_id
    zones = self.cf.zones.get(params=params)  # zones | pylint: disable=no-member
  File "/snap/certbot-dns-cloudflare/current/lib/python3.8/site-packages/CloudFlare/cloudflare.py", line 622, in get
    return self._base.call_with_auth('GET', self._parts,
  File "/snap/certbot-dns-cloudflare/current/lib/python3.8/site-packages/CloudFlare/cloudflare.py", line 80, in call_with_auth
    return self._call(method, headers, parts,
  File "/snap/certbot-dns-cloudflare/current/lib/python3.8/site-packages/CloudFlare/cloudflare.py", line 473, in _call
    raise CloudFlareAPIError(code, message, error_chain)
CloudFlare.exceptions.CloudFlareAPIError: Invalid request headers

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/snap/certbot/1042/bin/certbot", line 8, in <module>
    sys.exit(main())
  File "/snap/certbot/1042/lib/python3.8/site-packages/certbot/main.py", line 15, in main
    return internal_main.main(cli_args)
  File "/snap/certbot/1042/lib/python3.8/site-packages/certbot/_internal/main.py", line 1421, in main
    return config.func(config, plugins)
  File "/snap/certbot/1042/lib/python3.8/site-packages/certbot/_internal/main.py", line 1301, in certonly
    lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
  File "/snap/certbot/1042/lib/python3.8/site-packages/certbot/_internal/main.py", line 134, in _get_and_save_cert
    lineage = le_client.obtain_and_enroll_certificate(domains, certname)
  File "/snap/certbot/1042/lib/python3.8/site-packages/certbot/_internal/client.py", line 441, in obtain_and_enroll_certificate
    cert, chain, key, _ = self.obtain_certificate(domains)
  File "/snap/certbot/1042/lib/python3.8/site-packages/certbot/_internal/client.py", line 374, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/snap/certbot/1042/lib/python3.8/site-packages/certbot/_internal/client.py", line 421, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, best_effort)
  File "/snap/certbot/1042/lib/python3.8/site-packages/certbot/_internal/auth_handler.py", line 70, in handle_authorizations
    resps = self.auth.perform(achalls)
  File "/snap/certbot/1042/lib/python3.8/site-packages/certbot/plugins/dns_common.py", line 57, in perform
    self._perform(domain, validation_domain_name, validation)
  File "/snap/certbot-dns-cloudflare/current/lib/python3.8/site-packages/certbot_dns_cloudflare/_internal/dns_cloudflare.py", line 77, in _perform
    self._get_cloudflare_client().add_txt_record(domain, validation_name, validation, self.ttl)
  File "/snap/certbot-dns-cloudflare/current/lib/python3.8/site-packages/certbot_dns_cloudflare/_internal/dns_cloudflare.py", line 107, in add_txt_record
    zone_id = self._find_zone_id(domain)
  File "/snap/certbot-dns-cloudflare/current/lib/python3.8/site-packages/certbot_dns_cloudflare/_internal/dns_cloudflare.py", line 201, in _find_zone_id
    raise errors.PluginError('Error determining zone_id: {0} {1}. Please confirm '
certbot.errors.PluginError: Error determining zone_id: 6003 Invalid request headers. Please confirm that you have supplied valid Cloudflare API credentials. (Did you copy your entire API token/key? To use Cloudflare tokens, you'll need the python package cloudflare>=2.3.1. This certbot is running cloudflare 2.3.1)
2021-03-23 23:59:14,145:ERROR:certbot._internal.log:Error determining zone_id: 6003 Invalid request headers. Please confirm that you have supplied valid Cloudflare API credentials. (Did you copy your entire API token/key? To use Cloudflare tokens, you'll need the python package cloudflare>=2.3.1. This certbot is running cloudflare 2.3.1)

Here is the relevant nginx server block or Apache virtualhost for the domain I am configuring:

N/A - Only running certbot without any webserver for DNS validation.

ZetaTwo avatar Mar 23 '21 23:03 ZetaTwo

This works okay for me on the latest snap:

certbot-dns-cloudflare    1.13.0                      631    latest/stable    certbot-eff✓        -

I tested with the most narrow scope, just DNS:Edit on the zone itself.

Could you check whether maybe you are being affected by this behavior: https://requests.readthedocs.io/en/master/user/authentication/#netrc-authentication?

alexzorin avatar Mar 23 '21 23:03 alexzorin

Hmm, ok I have an idea what it might be. I think I was trying to generate a wildcard certificate for sub1.example.com while at the same time having an NS record set up for sub1.example.com in the Cloudflare settings. I will investigate further.

ZetaTwo avatar Mar 24 '21 00:03 ZetaTwo

Anything new here? I think i got the same issue.

I am trying to setup a haproxy via docker with certbot.

Setup for certbot:

pip install certbot certbot-dns-cloudflare cloudflare

2021-07-13 17:42:29,741:INFO:certbot._internal.auth_handler:Performing the following challenges:
2021-07-13 17:42:29,741:INFO:certbot._internal.auth_handler:dns-01 challenge for example.com
2021-07-13 17:42:29,741:INFO:certbot._internal.auth_handler:dns-01 challenge for www.example.com
2021-07-13 17:42:29,744:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.cloudflare.com:443
2021-07-13 17:42:30,732:DEBUG:urllib3.connectionpool:https://api.cloudflare.com:443 "GET /client/v4/zones?name=example.com&per_page=1 HTTP/1.1" 400 None
2021-07-13 17:42:30,734:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/certbot_dns_cloudflare/_internal/dns_cloudflare.py", line 187, in _find_zone_id
    zones = self.cf.zones.get(params=params)  # zones | pylint: disable=no-member
  File "/usr/lib/python3.9/site-packages/CloudFlare/cloudflare.py", line 672, in get
    return self._base.call_with_auth('GET', self._parts,
  File "/usr/lib/python3.9/site-packages/CloudFlare/cloudflare.py", line 126, in call_with_auth
    return self._call(method, headers, parts,
  File "/usr/lib/python3.9/site-packages/CloudFlare/cloudflare.py", line 498, in _call
    raise CloudFlareAPIError(code, message, error_chain)
CloudFlare.exceptions.CloudFlareAPIError: Invalid request headers

certbot certonly \
  -n --agree-tos --email [email protected] \
  --staging \
  --non-interactive \
  --test-cert \
  --dns-cloudflare \
  --dns-cloudflare-credentials /usr/local/etc/haproxy/cloudflare.ini \
  --config-dir /usr/local/etc/haproxy/certbot/config/ \
  --work-dir /usr/local/etc/haproxy/certbot/workdir/ \
  --logs-dir /usr/local/etc/haproxy/certbot/logs/ \
  -d example.com \
  -d www.example.com

Yet if I call /client/v4/zones?name=example.com&per_page=1 manual with bearer token I do get a result.

edit:

Forgot the versions:

certbot==1.17.0
certbot-dns-cloudflare==1.17.0
cloudflare==2.8.15

michaelsstuff avatar Jul 13 '21 17:07 michaelsstuff

use dns_cloudflare_api_token instead of dns_cloudflare_api_key in your cloudflare.ini

kcepaxe avatar May 16 '22 09:05 kcepaxe

Getting this same issue using the certbot/dns-cloudflare docker image.

Error determining zone_id: 6003 Invalid request headers. Please confirm that you have supplied valid Cloudflare API credentials. (Did you copy your entire API token/key? To use Cloudflare tokens, you'll need the python package cloudflare>=2.3.1. This certbot is running cloudflare 2.11.7)

Using a global API key + email works fine but changing to an api token (verified as working many times) just throws this same error. Tried with both the latest build and the nightly.

Actual command being run:

certbot certonly \
    --dns-cloudflare \
    --dns-cloudflare-credentials /cloudflare.ini \
    --agree-tos \
    --no-eff-email \
    -n \
    -m $CLOUDFLARE_EMAIL \
    -d $CLOUDFLARE_DOMAIN_LIST

And /cloudflare.ini format:

dns_cloudflare_api_token = $CLOUDFLARE_TOKEN

snk-nick avatar Sep 03 '23 12:09 snk-nick