DNS Challenge with Cloudflare fails

[dkhelms]

• Have you pulled and found the error with jc21/nginx-proxy-manager:latest docker image?
  • Yes
• Are you sure you're not using someone else's docker image?
  • Yes
• Have you searched for similar issues (both open and closed)?
  • Yes

Describe the bug I try to use DNS Challenge with Cloudflare to get a cert but it doesn't work.

Nginx Proxy Manager Version 2.10.4

To Reproduce

1. Go to SSL Certificates
2. Click Add New SSL Certificate
3. Choose Let's Encrypt
4. Use DNS Challenge and Cloudflare as DNS Provider

Expected behavior For a cert to be issued.

Screenshots

Operating System I am using Ubuntu 22.04 with the newest version of Portainer

Additional context Here are the errors:

Error: Command failed: . /opt/certbot/bin/activate && pip install --no-cache-dir certbot-dns-cloudflare==$(certbot --version | grep -Eo '[0-9](\.[0-9]+)+') cloudflare && deactivate
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-cloudflare/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-cloudflare/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-cloudflare/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-cloudflare/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-cloudflare/
ERROR: Could not find a version that satisfies the requirement certbot-dns-cloudflare==2.5.0 (from versions: none)
ERROR: No matching distribution found for certbot-dns-cloudflare==2.5.0

    at ChildProcess.exithandler (node:child_process:402:12)
    at ChildProcess.emit (node:events:513:28)
    at maybeClose (node:internal/child_process:1100:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)```

[SamBouwer]

@dkhelms maybe a stupid question, but did you update the API key in the example that is shown when you select the Cloudflare DNS provider from the dropdown list? If you did update that key with a valid key from your Cloudflare account, can you resolve api.cloudflare.com (for example with "nslookup api.cloudflare.com"?

[dkhelms]

@dkhelms maybe a stupid question, but did you update the API key in the example that is shown when you select the Cloudflare DNS provider from the dropdown list? If you did update that key with a valid key from your Cloudflare account, can you resolve api.cloudflare.com (for example with "nslookup api.cloudflare.com"?

Yes, I updated the key as well. The problem was the newest version of NPM. I had to go back to 2.9.14 to renew the cert and make everything start working again.

[Doutianbao]

@dkhelms maybe a stupid question, but did you update the API key in the example that is shown when you select the Cloudflare DNS provider from the dropdown list? If you did update that key with a valid key from your Cloudflare account, can you resolve api.cloudflare.com (for example with "nslookup api.cloudflare.com"?

Yes, I updated the key as well. The problem was the newest version of NPM. I had to go back to 2.9.14 to renew the cert and make everything start working again.

same bug on v2.10.4

[deadblue0910]

@dkhelms也许是一个愚蠢的问题，但是当您从下拉列表中选择 Cloudflare DNS 提供商时，您是否更新了示例中显示的 API 密钥？如果您确实使用 Cloudflare 帐户中的有效密钥更新了该密钥，您能否解析 api.cloudflare.com（例如使用“nslookup api.cloudflare.com”？

是的，我也更新了密钥。问题出在最新版本的 NPM 上。我必须返回 2.9.14 更新证书并使一切重新开始工作。

v2.10.4 上有同样的错误

Have you solved this problem?also v2.10.4

[dkhelms]

@dkhelms也许是一个愚蠢的问题，但是当您从下拉列表中选择 Cloudflare DNS 提供商时，您是否更新了示例中显示的 API 密钥？如果您确实使用 Cloudflare 帐户中的有效密钥更新了该密钥，您能否解析 api.cloudflare.com（例如使用“nslookup api.cloudflare.com”？

是的，我也更新了密钥。问题出在最新版本的 NPM 上。我必须返回 2.9.14 更新证书并使一切重新开始工作。

v2.10.4 上有同样的错误

Have you solved this problem?also v2.10.4

Sorry, but I never did.

[jc21]

You are welcome to try the github-develop docker tag, it's bleeding edge and frankly, I need people to test more DNS providers that I don't use.

It has certbot v2.8.0 (previously was v2.5.0) and also means DNS plugins will be using v2.8.0 as well.

[moviemakr1620]

@jc21 How do you do that? right now mine is having the same issues and would try anything.

[Wamy-Dev]

same issue for me, using the docker container.

[MattXcz]

+1 on this... using proxmox lxc, first try failed, second one finished without errors...

[hurz07]

@jc21 - tried GitHub-develp tag, no change for me. Currently running v2.11.1.

I use duckdns.org and run nginxproxymanager in a docker container on synology using portainer. I created a macvlan network and excluded IPv6. Everything runs well except creating lets encrypt certificates with duckdns DNS-challenge.

The error message:

CommandError: WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
ERROR: Could not find a version that satisfies the requirement certbot-dns-duckdns~=0.9 (from versions: none)
ERROR: No matching distribution found for certbot-dns-duckdns~=0.9

    at /app/lib/utils.js:16:13
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:518:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)

Looking forward for hints or an solution :-) Thank you in advance.

[MingfuYAN]

I solved the problem. My cause was that DNSSEC was not configured correctly, visit this site to see if DNSSEC is configured correctly.

I found this issue by looking directly at the log.

1. docker exec -it NPM-container-name bash
2. cat /tmp/letsencrypt-log/letsencrypt.log

[hurz07]

I also tested it in a clean virtual machine with an "own" IP address and it worked. Think my problem is the use of "macvlan" in docker.

Thank you for your nice work!

@jc21 - tried GitHub-develp tag, no change for me. Currently running v2.11.1.

I use duckdns.org and run nginxproxymanager in a docker container on synology using portainer. I created a macvlan network and excluded IPv6. Everything runs well except creating lets encrypt certificates with duckdns DNS-challenge.

The error message:

CommandError: WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
ERROR: Could not find a version that satisfies the requirement certbot-dns-duckdns~=0.9 (from versions: none)
ERROR: No matching distribution found for certbot-dns-duckdns~=0.9

    at /app/lib/utils.js:16:13
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:518:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)

Looking forward for hints or an solution :-) Thank you in advance.

[Karstenve]

I had the same issue and found a lot of open or stale issues around this repo. What I found is that when I tried to manually install the certbot-dns-cloudflare when executing a bash in the docker container, for some reason the container couldn't reach the appropriate packages. What I did is add "network-mode: host" to the docker compose file and after that I could manually install and get the certificate working. I hope this helps people. I'm not going to react to other issues so I hope people find this.

EDIT: After some more experimenting I found out the npm container didn't have internet access. After looking into the DNS config I found out Tailscale had replaced the DNS info in /etc/resolv.conf which caused DNS issues in all my containers. After fixing this file by disabling the replacement done by tailscale I could rebuild the containers and the issue was fixed. Adding network-mode: host also works but it's not ideal as I needed a bridged network in this case.

[Waldorf3]

Getting similar errors here. Suddenly certs stopped being renewed. When trying to renew manually getting this error:

Internal Error
CommandError: Saving debug log to /tmp/letsencrypt-log/letsencrypt.log
/opt/certbot/lib/python3.11/site-packages/certbot_dns_cloudflare/_internal/dns_cloudflare.py:107: PendingDeprecationWarning: 
       !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!   WARNING  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
       !! You're seeing this warning because you've upgraded the Python package 'cloudflare' to version  !!
       !! 2.20.* via an automated upgrade without version pinning. Version 2.20.0 exists to catch any    !!
       !! of these upgrades before Cloudflare releases a new major release under the release number 3.x. !!
       !!                                                                                                !!
       !! Should you determine that you need to revert this upgrade and pin to v2.19.* it is recommended !!
       !! you do the following: pip install --upgrade cloudflare==2.19.* or equivilant.                  !!
       !!                                                                                                !!
       !! Or you can upgrade to v3.x. NOTE: Release 3.x will not be code-compatible or call-compatible   !!
       !! with previous releases. To see more about upgrading to next major version, please see:         !!
       !! https://github.com/cloudflare/python-cloudflare/discussions/191                                !!
       !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
  self.cf = CloudFlare.CloudFlare(token=api_token)
/opt/certbot/lib/python3.11/site-packages/certbot_dns_cloudflare/_internal/dns_cloudflare.py:107: PendingDeprecationWarning: 
       !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!   WARNING  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
       !! You're seeing this warning because you've upgraded the Python package 'cloudflare' to version  !!
       !! 2.20.* via an automated upgrade without version pinning. Version 2.20.0 exists to catch any    !!
       !! of these upgrades before Cloudflare releases a new major release under the release number 3.x. !!
       !!                                                                                                !!
       !! Should you determine that you need to revert this upgrade and pin to v2.19.* it is recommended !!
       !! you do the following: pip install --upgrade cloudflare==2.19.* or equivilant.                  !!
       !!                                                                                                !!
       !! Or you can upgrade to v3.x. NOTE: Release 3.x will not be code-compatible or call-compatible   !!
       !! with previous releases. To see more about upgrading to next major version, please see:         !!
       !! https://github.com/cloudflare/python-cloudflare/discussions/191                                !!
       !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
  self.cf = CloudFlare.CloudFlare(token=api_token)
Error determining zone_id: 6003 Invalid request headers. Please confirm that you have supplied valid Cloudflare API credentials. (Did you copy your entire API token/key? To use Cloudflare tokens, you'll need the python package cloudflare>=2.3.1. This certbot is running cloudflare 2.20.0)
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/letsencrypt-log/letsencrypt.log or re-run Certbot with -v for more details.

    at /app/lib/utils.js:16:13
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:519:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)

Don't know anything about updating python, this is a docker container, just pulled the latest available.

[Framdark]

I am also having this issue error logs say this

`Error: Command failed: certbot certonly --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-25" --agree-tos --email "[email protected]" --domains "*.domain.top,domain.top" --authenticator dns-cloudflare --dns-cloudflare-credentials "/etc/letsencrypt/credentials/credentials-25" Saving debug log to /tmp/letsencrypt-log/letsencrypt.log Some challenges have failed. Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/letsencrypt-log/letsencrypt.log or re-run Certbot with -v for more details.

at ChildProcess.exithandler (node:child_process:422:12)
at ChildProcess.emit (node:events:517:28)
at maybeClose (node:internal/child_process:1098:16)
at ChildProcess._handle.onexit (node:internal/child_process:303:5)`

[paradox1612]

I had the same issue, Need to reinstall pip and pip install cloudflare==2.19.*

https://blog.thekush.dev/how-to-fix-nginx-manager-certbot_dns_cloudflare-_internal-dns_cloudflare-plugin-error/

[Framdark]

I can't do that because I am running it through hass

On Fri, Jun 28, 2024 at 10:17 PM paradox1612 @.***> wrote:

I had the same issue, Need to reinstall pip and pip install cloudflare==2.19.*

https://blog.thekush.dev/how-to-fix-nginx-manager-certbot_dns_cloudflare-_internal-dns_cloudflare-plugin-error/

— Reply to this email directly, view it on GitHub https://github.com/NginxProxyManager/nginx-proxy-manager/issues/3305#issuecomment-2197874855, or unsubscribe https://github.com/notifications/unsubscribe-auth/AM545AWXP5665XGGHWLLTYTZJYRONAVCNFSM6AAAAAA7AEBQPWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJXHA3TIOBVGU . You are receiving this because you commented.Message ID: @.***>

--

http://www.sigbee.com?utm_campaign=signature&origUserUID=34a580e69f034c7a45223baafe78379d1087bbd6&origSigUserImage=a829529e4fc9eafc.gif

[caiocesarstx]

I had the same problem with Cloudflare plugin:

using NPM on Docker, with docker swarm managing it and using DNS to access the NPM instance.

The log is below:

2024-07-01 20:05:38,016:DEBUG:certbot._internal.log:Exiting abnormally: Traceback (most recent call last): File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/plugins/disco.py", line 191, in find_all cls._load_entry_point(entry_point, plugins) File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/plugins/disco.py", line 203, in _load_entry_point plugin_ep = PluginEntryPoint(entry_point) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/plugins/disco.py", line 42, in init self.plugin_cls: Type[interfaces.Plugin] = entry_point.load() ^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/importlib/metadata/init.py", line 202, in load module = import_module(match.group('module')) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1206, in _gcd_import File "", line 1178, in _find_and_load File "", line 1149, in _find_and_load_unlocked File "", line 690, in _load_unlocked File "", line 940, in exec_module File "", line 241, in _call_with_frames_removed File "/opt/certbot/lib/python3.11/site-packages/certbot_dns_cloudflare/_internal/dns_cloudflare.py", line 9, in import CloudFlare ModuleNotFoundError: No module named 'CloudFlare'

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/opt/certbot/bin/certbot", line 8, in sys.exit(main()) ^^^^^^ File "/opt/certbot/lib/python3.11/site-packages/certbot/main.py", line 19, in main return internal_main.main(cli_args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/main.py", line 1841, in main plugins = plugins_disco.PluginsRegistry.find_all() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/plugins/disco.py", line 193, in find_all raise errors.PluginError( certbot.errors.PluginError: The 'certbot_dns_cloudflare._internal.dns_cloudflare' plugin errored while loading: No module named 'CloudFlare'. You may need to remove or update this plugin. The Certbot log will contain the full error details and this should be reported to the plugin developer. 2024-07-01 20:05:38,016:ERROR:certbot._internal.log:The 'certbot_dns_cloudflare._internal.dns_cloudflare' plugin errored while loading: No module named 'CloudFlare'. You may need to remove or update this plugin. The Certbot log will contain the full error details and this should be reported to the plugin developer.

[paradox1612]

you can do : pip install --upgrade cloudflare==2.19.*

[Framdark]

I unfortunately can't use pip because I am running NPM in home assistant.

[caiocesarstx]

you can do : pip install --upgrade cloudflare==2.19.*

It worked, thanks!

[7/1/2024] [9:16:54 PM] [Certbot ] › ▶ start Installing cloudflare... ***** omited ******* Waiting 120 seconds for DNS changes to propagate Successfully received certificate. Certificate is saved at: /etc/letsencrypt/live/npm-52/fullchain.pem Key is saved at: /etc/letsencrypt/live/npm-52/privkey.pem This certificate expires on 2024-09-29. These files will be updated when the certificate renews. NEXT STEPS:

• The certificate will need to be renewed before it expires. Certbot can automatically renew the certificate in the background, but you may need to take steps to enable that functionality. See https://certbot.org/renewal-setup for instructions.

[caiocesarstx]

I unfortunately can't use pip because I am running NPM in home assistant.

Is importante to say that, this works for me, but for the running container, if eventually needs to restart, probably will be an error on renew and need to do it again for other certificates.

[Shredder5262]

pip install cloudflare==2.19.*

this worked for me

[claffern]

I had the same issue, Need to reinstall pip and pip install cloudflare==2.19.*

https://blog.thekush.dev/how-to-fix-nginx-manager-certbot_dns_cloudflare-_internal-dns_cloudflare-plugin-error/

this worked for me!!!!

[Silicon51]

So, there's a chance that you have my case: both piHole and NPM as docker containers. Due to some weird behavior of DNS resolver NPM container do not have access to internet so cannot request for cert. For me it log errors like Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/cloudflare/ Also I have error Failed to check the reachability due to a communication error with site24x7.com nginx proxy when in version 2.12.3 I use option "Test Server Reachability" from tab SSL Certificates. How to solve it? add following to your NPM docker compose:

dns:
  - 172.19.0.4 <<pihole IP adress>>
  - 1.1.1.1
  - 8.8.8.8

[Kalkoen31]

I had similar errors. Adding a DNS in the docker-compose fixed it. Thanks @Silicon51