Support wildcard certificates with ACMEv2

[raqbit]

trafficstars

On January 27 letsencrypt will make star certificates fully available via the ACME v2 protocol. Are you planning on adding support for this?

[buchdag]

Wilcard certs will require dns-01 challenge and simp_le only support http-01, so they won't be available through letsencrypt-nginx-proxy-companion.

https://github.com/zenhack/simp_le/issues/101

[raqbit]

Well that's a bummer. As Python-acme is part of certbot I'd guess they will implement v2 soon-ish, and then simple_le will able to support it as well in the future. Anyways, thanks!

[buchdag]

Unfortunately it's not as simple as that, let me clarify : support for ACME V2 will be implemented in simp_le, but dns-01 challenge won't.

dns-01 challenge is not new or specific to ACME v2 and the original simp_le developer (@kuba) made the explicit choice to limit simp_le to http-01 challenge from the beginning (you can see it in the manifesto, point 3).

[raqbit]

Oh, I see. Thanks for helping :)

[buchdag]

@Raqbit I'm re-opening so the issue and the reason why we won't (/ can't) support wildcard certs have better visibility.

[almereyda]

Please see https://github.com/zenhack/simp_le/issues/101#issuecomment-372773310 for updates on the upstream issue, since the v2 endpoint has finally been released to production.

[buchdag]

@almereyda as told in my first message on this issue and on the simp_le issue you mentioned, wilcard certificates requires dns-01 challenge support, which is absent from simp_le (and there are no plans to add it).

This is not the same thing as (or related to) Acme v2 support.

[Dragnucs]

I am curious. Can't simp_le be replaced with another components that could handle dns-01 challenge? If not, what are our options? Can't we hack the configuration a bit on our end to support it?

[buchdag]

It could, letsencrypt-nginx-proxy-companion is pretty much "just" bash automation around simp_le and nginx-proxy, there is nothing preventing someone from re-writting it to use another ACME client and provide additional features. As usual with small open source projects the only real issues are the amount of work necessary and the time it takes.

I've been thinking for a while about doing a fork of this project using certbot instead of simp_le but it's pretty far from a drop-in replacement as the two behave in very different ways. I lack the time to do more than experiment a bit with the idea once in a while.

As for other options, the most serious is probably switching to a bigger project like https://github.com/containous/traefik

To be honest I have my doubts about the long term viability of the nginx-proxy / docker-gen solution as it's now extremely hard to get contributions merged to those projects (mostly due to the fact that their creator is very busy working at influxdb, I guess. That's understandable).

[Dragnucs]

@buchdag thank you for your reply.

[almereyda]

The simp_le issue mentions that they are planning to support the DNS-01 challenge soon'ish.

On 15 March 2018 at 10:51, Dragnucs [email protected] wrote:

@buchdag https://github.com/buchdag thank you for your reply.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/JrCs/docker-letsencrypt-nginx-proxy-companion/issues/319#issuecomment-373319869, or mute the thread https://github.com/notifications/unsubscribe-auth/ABka_GISKEcuTX9F-FpUc2P_lNQPU3rIks5tejmzgaJpZM4RmAk2 .

[buchdag]

Errr no it does not.

We (zenhack and I) said on that issue that ACME v2 support is planned (we said nothing about when, and it might turn out to be harder than expected) and that dns-01 is out of scope.

Again dns-01 challenge is required to obtain a wildcard certificate through the Let's Encrypt ACME v2 endpoint, but they are not one and the same thing: implementing ACME v2 does not imply implementing dns-01 challenge at all.

[evertramos]

@buchdag Thanks for your contribution!!! Traefik works really fine with it!!! I am doing some tests around it and it seems pretty stable as well. Since I have a couple servers running this repo It will take a couple months to release something usefull, but thank in advance!

[BirkhoffLee]

Maybe give users a hint in https://github.com/JrCs/docker-letsencrypt-nginx-proxy-companion/blob/master/app/letsencrypt_service#L106 when they receive this useless error? Like adding the link to this issue in the output. We expect this to work, but it actually fails

nginx-letsencrypt    | /etc/nginx/certs/*.birkhoff.me /app
nginx-letsencrypt    | Creating/renewal *.birkhoff.me certificates... (*.birkhoff.me)
nginx-letsencrypt    | /usr/lib/python2.7/site-packages/acme/jose/jwa.py:110: CryptographyDeprecationWarning: signer and verifier have been deprecat
ed. Please use sign and verify instead.
nginx-letsencrypt    |   signer = key.signer(self.padding, self.hash)
nginx-letsencrypt    | ACME server returned an error: urn:acme:error:malformed :: The request message was malformed :: Error creating new authz :: W
ildcard names not supported

It actually took me a while to figure out what happened. Thanks for the nice work tho

[buchdag]

@BirkhoffLee "expect this to work" "useless error" ... nice tone there.

BTW the "useless error" actually comes from simp_le, not from this container's code.

[BirkhoffLee]

I’m sorry, I didn’t mean to say that. I’m not a native English speaker so.. haha

I’ll try to dig into the code and see if I can add a hint to the error when I have free time :)

Sorry again for what I said, and thanks to everyone for the contribution to the awesome project!

On Wed, Apr 25, 2018 at 2:23 PM Nicolas Duchon [email protected] wrote:

@BirkhoffLee https://github.com/BirkhoffLee "expect this to work" "useless error" ... nice tone there.

BTW the "useless error" actually comes from simp_le, not from this container's code.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/JrCs/docker-letsencrypt-nginx-proxy-companion/issues/319#issuecomment-384173713, or mute the thread https://github.com/notifications/unsubscribe-auth/AHxUf9jjsuyAqnU95EOfvJuJM1sQCOPEks5tsBZ9gaJpZM4RmAk2 .

[buchdag]

My bad, I did not realize you were a non native english speaker. 😅

You're right with the fact that some warnings about unsupported features (as of now) like ACME V2 endpoints and wildcard certificates should be added. I'll try to add this to dev this week.

[BirkhoffLee]

Thanks 😄

On Wed, Apr 25, 2018 at 2:50 PM Nicolas Duchon [email protected] wrote:

My bad, I did not realize you were a non native english speaker. 😅

You're right with the fact that some warnings about unsupported features (as of now) like ACME V2 endpoints and wildcard certificates should be added. I'll try to add this to dev this week.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/JrCs/docker-letsencrypt-nginx-proxy-companion/issues/319#issuecomment-384178784, or mute the thread https://github.com/notifications/unsubscribe-auth/AHxUf2i0qWuzak52cgRseaj98kZ9vYikks5tsByhgaJpZM4RmAk2 .

[xiaods]

hi team, how long we can support ACME v2 feature?

[whlsxl]

@buchdag actually traefik use lego as LE lib

[ejbp]

While wildcard is not implemented, there's any workaround to minor work effort on setting up a wildcard configuration on nginx with all the other benefits of this image? e.g. Can I manually generate a wildcard certificate ant let know "letsencrypt-nginx-proxy-companion" to use these custom certificates instead of try to create/renew new ones?

I tried to have my custom certificates on certificates path but they are still replaced by the single domain certificate that "letsencrypt-nginx-proxy-companion" generates. I can in a ugly way limit the permissions to read only for my custom certificates but still, the let's encrypt api will be called if I need to restart the docker which may enter in a out of limit requests to letsencrypt.

Meanwhile, it would be great if this will be implemented shortly or if I really need to consider other solution :(

Thanks in advance.

[buchdag]

letsencrypt-nginx-proxy-companion does not "use" certificates. It handle their creation, renewal, and some symlink work so that nginx-proxy can pick them up.

The application you want to use your externally obtained certificates with is nginx-proxy and that amount to following their existing doc and doing a naming or symlinking job similar to what the companion do.

The companion should not be involved at all when you use externally obtained wildcard certs.

[ejbp]

I understand your point and makes sense. Though, I'm not able to have a mixed configuration with domains that are under letsencrypt-nginx-proxy-companion management and domains that are not, having in mind that jwilder/nginx-proxy is the same and using the same certs dir for all domains. What's happening is that letsencrypt-nginx-proxy-companion is removing my custom certificates since it considers that they are unsed.

log:

letsencrypt-nginx-proxy-companion    | Symlinked domains: yyy.com xxx.com
letsencrypt-nginx-proxy-companion    | Enabled domains: xxx.com
letsencrypt-nginx-proxy-companion    | Disabled domains: yyy.com
letsencrypt-nginx-proxy-companion    | Some domains are disabled. Check them to remove unused symlinks.
letsencrypt-nginx-proxy-companion    | 
letsencrypt-nginx-proxy-companion    | Checking domain yyy.com: 
letsencrypt-nginx-proxy-companion    | Checking yyy.com.crt - removing.
letsencrypt-nginx-proxy-companion    | Checking yyy.com.key - removing.
letsencrypt-nginx-proxy-companion    | Checking yyy.com.dhparam.pem
letsencrypt-nginx-proxy-companion    | Checking yyy.com.chain.pem - removing.
letsencrypt-nginx-proxy-companion    | Unused domains checking is finished.

docker-compose.yml:

services:
  letsencrypt_nginx_proxy_companion:
    image: jrcs/letsencrypt-nginx-proxy-companion
    container_name: letsencrypt-nginx-proxy-companion
    volumes_from:
      - load_balancer
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "${LETSENCRYPT_CERTS_PATH}:/etc/nginx/certs:rw"
    environment:
      - NGINX_DOCKER_GEN_CONTAINER=load_balancer
  load_balancer:
    image: jwilder/nginx-proxy
    container_name: load_balancer
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./docker/load-balancer/nginx-proxy-vhost.d:/etc/nginx/vhost.d
      - /var/run/docker.sock:/tmp/docker.sock:ro
      - /usr/share/nginx/html
      - "${LETSENCRYPT_CERTS_PATH:/etc/nginx/certs:ro"
    labels:
      com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy: "true"
    environment:
      - ENABLE_IPV6=true
    restart: always
  website:
    container_name: website
    image:  ${IMAGE_WEBSITE}
    ports:
      - 5555:80
    environment:
      - VIRTUAL_HOST=xxx.com
      - LETSENCRYPT_HOST=xxx.com
      - [email protected]
  custom_certificates_website:
    container_name: custom-certificates-website
    image:  ${IMAGE_CUSTOM_CERTIFICATES_WEBSITE}
    ports:
      - 5556:80
    environment:
      - VIRTUAL_HOST=yyy.com,*.yyy.com

[buchdag]

Ok that's a bug, it should not happen. Do you mind opening a separate issue so we can work toward fixing it ?

[speshak]

FWIW: I forked this a while back to use acme.sh instead of simp_le explicitly so I could do dns-01 validation via AWS Route53. I haven't tried using wildcard certs with my fork, but if someone really wanted to they could try: https://github.com/speshak/docker-letsencrypt-nginx-proxy-companion/tree/acme.sh

[buchdag]

A v2.0 version using acme.sh is being worked on and will be available at least on the dev branch by the end of march (that's an estimate, not a hard deadline).

This new version will be able to use ACME v2 endpoint (and will probably default to them) and to issue ECDSA certificates but I'm not sure yet if the ability to issue wildcard certificate will be present as soon as v2.0 or on a later version.

Some feature will have to be handled very differently than on the current version (namely pretty much everything about ACME accounts and emails), the container will require some additional configuration and all existing certs will have to be issued again (with new domain authorisations).

I've considered and tested lego as suggested by @whlsxl but it does not fit as well as acme.sh and require too much work (it pretty much require to build all the new / existing / changed / renewed cert logic from the ground up).

Also considering(ed) dehydrated but it does not have built in DNS manipulation and would require pairing with something like lexicon to enable automated wildcard certificate issuance.

[Greek64]

@buchdag I was about to make of "proof of concept" PR where I swap simp_le for acme.sh, but as I see you are already working on integrating acme.sh. I would suggest that the first version only swaps the client and does only ensure backwards compatibility, before adding new features like ECDSA, stapling, wildcard, etc... Will the acme client be selectable (e.g. via an environmental variable) or will the project as a whole move to the new client?

[buchdag]

Backward compatibility might be really hard depending on what you expect to be backward compatible, ie most environment variables and config parameters won't change, but reusing existing data will be either hard (certificates and private key) or very hard / not possible (ACME account keys) as simp_le and acme.sh store them in very different ways.

[buchdag]

I'll open another separate issue with more details about were I am now with the acme.sh version and what the issues / challenges are so we can discuss this as soon as I'll have some time.

[maximkrusina]

In between, I found this - I hope it will help someone: https://github.com/adferrand/docker-letsencrypt-dns