.well-known not passing?

[thiggin]

Trying to proxy for MTA-STS config but when the server is behind nginx proxy manager it is not passing request for anything in the .well-known directory to the back end server. I am sure this probably has something to do with config for the Let's Encrypt stuff to work but is also very limiting in what we can host behind the nginx proxy manager. Trying to run this all through the proxy for a central place to manage all Let's Encrypt certs. Is this expected behavior or something that can be addresses with custom config or something?

[joshbenner]

The nginx configs that it generates seem to always include the letsencrypt configurations, which will swallow any request to /.well-known/acme-challenge/*.

This is important for proxy manager's ability to handle Let's Encrypt HTTP challenges; however, a simple toggle option could be presented in the host config to disable Let's Encrypt on that host, thereby causing the include to not be written in that host's config.

The impact would be that any hostname associated with that config could not acquire or renew any Let's Encrypt certificates.

[thecybermind]

Assuming you don't actually need NPM to manage any certs via HTTP (I use the DNS option to get wildcard certs for the various domains pointing at NPM, but one of the servers behind NPM needs to get its own certs for use with non-HTTP services), the simplest way to temporarily handle this is to change docker-compose.yml to include this in the volumes section:

- ./data/nginx/letsencrypt-acme-challenge.conf:/etc/nginx/conf.d/include/letsencrypt-acme-challenge.conf

And then include a blank letsencrypt-acme-challenge.conf file in ./data/nginx/ (based on where docker-compose.yml is).

This fully-reversible change will basically block any of the /.well-known/-handling code from being added to the nginx config for any host, which will have the result of passing all /.well-known/ requests through to the servers behind NPM.

If you do need NPM to get any cert via HTTP, you would probably need to add custom locations to those hosts using the code from the original /etc/nginx/conf.d/include/letsencrypt-acme-challenge.conf so that the request gets intercepted properly by NPM.

[DieserMerlin]

My solution for this problem is creating a common volume for the .well-known folder:

docker volume create well-known

Map this to the .well-known folder in NPM by adding a volume to the compose-file:

services:
  npm:
    ...
    volumes:
      - well-known:/data/letsencrypt-acme-challenge/.well-known
    ...
    
volumes:
  well-known:
    external: true

Now, map this volume onto any .well-known folder in any container. I use this for my poste.io mailserver, where I map the volume to /opt/www/.well-known to enable poste.io to get a cert for my e-mails.

[EnfermeraSexy]

Having the same problem with poste.io mailserver. There should be an option to allow passthrought for subdomain/.well-known/acme-challenge/*, and more when you are using dns challenge for npm.

[DieserMerlin]

@EnfermeraSexy Have you tried my solution?

[EnfermeraSexy]

@EnfermeraSexy Have you tried my solution?

No, just mounted an empty file to /etc/nginx/conf.d/include/letsencrypt-acme-challenge.conf, but that is not a proper fix/solution.

[DieserMerlin]

If you map the same volume to both, NPM and the poste.io folder as described, the authorization works for both containers.

[rafpadilla]

My solution for this problem is creating a common volume for the .well-known folder:

docker volume create well-known

Map this to the .well-known folder in NPM by adding a volume to the compose-file:

services:
  npm:
    ...
    volumes:
      - well-known:/data/letsencrypt-acme-challenge/.well-known
    ...
    
volumes:
  well-known:
    external: true

Now, map this volume onto any .well-known folder in any container. I use this for my poste.io mailserver, where I map the volume to /opt/www/.well-known to enable poste.io to get a cert for my e-mails.

Thanks, that works for me!

[george-viaud]

Related problem, in that my proxy manager (running on a rasp-pi) forwards traffic to my mail server running iredadmin (a different rasp pi). All traffic to .well-known/acme-challenge/ is stopped by the Proxy Manager server and never makes it to the mail server.

Is there a way to allow the traffic to pass through normally and only divert the traffic to be handled by the Proxy Manager server when it's doing its own LetsEncrypt renewal?

Alternatively, is there a way to tell LetsEncrypt (certbot) on the mail server to use a custom acme-challenge path?

[DieserMerlin]

Have you tried my solution? Still works without any problems for me

[george-viaud]

Have you tried my solution? Still works without any problems for me

Thanks for the reply DieserMerlin - if I understand your solution correctly (I may not) it requires the Proxy Manager and mail server be on the same physical device? If not, I must not understand the solution correctly.

[DieserMerlin]

You must be able to use the same volume. There are ways to share a volume across multiple hosts, which is more complicated, but should make the solution work across multiple hosts with a little effort. For example, you could use the same ntfs network mount on both containers.

[george-viaud]

@DieserMerlin thanks for that. I solved my problem by deriving from your solution - I used SSH on the server I was running N.P.M. and the same on the mail server, then used scp on a cron to clone the certificate files and restart the necessary mail services every handful of days. I hope this helps someone else.

[evelynhathaway]

It sounds like we could disable the ACME responses from the NPM template based on the dns_challenge variable to resolve this issue.

{% if certificate.provider == "letsencrypt" %}
  # Let's Encrypt SSL
+ {% if !certificate.meta.dns_challenge %}
    include conf.d/include/letsencrypt-acme-challenge.conf;
+ {% endif %}
  include conf.d/include/ssl-ciphers.conf;
  ssl_certificate /etc/letsencrypt/live/npm-{{ certificate_id }}/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/npm-{{ certificate_id }}/privkey.pem;

https://github.com/NginxProxyManager/nginx-proxy-manager/blob/fd30cfe98bba365e9630a791bdc86f01c08c6fa1/backend/templates/_certificates.conf#L2-L4

Does anyone more familiar with the project want to confirm?

[evelynhathaway]

Also, if you cannot replace the templates or .well-known directory on your NPM filesystem, you can use this workaround instead:

Custom Nginx Configuration

rewrite ^(/.well-known/acme-challenge/.*)$ /internal$1 last;

location ~ ^/internal(/.well-known/acme-challenge/.*)$ {
  proxy_pass http://$server$1;
}

It works by rewriting the internal URI before the ACME challenge location can respond (related to #2418).

[aldington-david]

Also, if you cannot replace the templates or directory on your NPM filesystem, you can use this workaround instead:.well-known

Custom Nginx Configuration

rewrite ^(/.well-known/acme-challenge/.*)$ /internal$1 last;

location ~ ^/internal(/.well-known/acme-challenge/.*)$ {
  proxy_pass http://$server$1;
}

It works by rewriting the internal URI before the ACME challenge location can respond (related to #2418).

Excellent answer！

[github-actions[bot]]

Issue is now considered stale. If you want to keep it open, please comment :+1:

[evelynhathaway]

This issue is still occurring, and would make a good first issue if the effects of disabling the ACME responses when the site uses a certificate with DNS challenges is appropriate for a fix PR.