docker-flow-proxy-letsencrypt
docker-flow-proxy-letsencrypt copied to clipboard
Published
20 hours ago •
n1b0r
n1b0r
It's running certbot every 2 seconds
Even with abnormal exits, it can't run the certbot every 2 seconds as this 100% guarantees it's going to be rate-limited and never succeed.
2018-02-27 08:38:31,175:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:38:32,486:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:38:37,842:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:38:39,171:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:38:43,897:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:38:45,368:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:38:49,636:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:38:52,217:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:38:55,382:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:38:58,564:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:01,086:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:05,625:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:08,251:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:12,138:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:14,719:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:18,006:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:20,829:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:24,174:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:26,976:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:29,469:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:30,488:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:33,081:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:35,582:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:37,454:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:39,473:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:41,668:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:43,678:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:45,619:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:47,892:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:49,722:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:52,405:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:54,384:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:55,903:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:39:58,663:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:40:01,630:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:40:07,607:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:40:15,278:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:40:20,691:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:40:25,874:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:40:27,492:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:40:32,103:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:40:34,006:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:40:39,114:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:40:40,646:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:40:44,056:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:40:46,941:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:40:50,447:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:40:53,819:ERROR:certbot.log:Exiting abnormally:
2018-02-27 08:41:00,080:ERROR:certbot.log:Exiting abnormally:
Domain: www.example.com
Type: unauthorized
Detail: Invalid response from http://www.example.com/.well-known/acme-challenge/moy16SD5sYeMo_WjP2EVnIW93b5T9oVPXZp0H_yFoKs: "<html>M
<head><title>404 Not Found</title></head>M
<body bgcolor="white">M
<center><h1>404 Not Found</h1></center>M
<hr><center>"
To fix these errors, please make sure that your domain name was entered correctly and the DNS A/AAAA record(s) for that domain contain(s) the right IP address.
2018-02-27 08:35:38,178:INFO:certbot.auth_handler:Cleaning up challenges
2018-02-27 08:35:38,178:DEBUG:certbot.plugins.webroot:Removing /opt/www/.well-known/acme-challenge/uyJ3XliXAfwmzjFhmcmc_SLHR2t7c_TH0Nbxik1TU3k
2018-02-27 08:35:38,178:DEBUG:certbot.plugins.webroot:Removing /opt/www/.well-known/acme-challenge/moy16SD5sYeMo_WjP2EVnIW93b5T9oVPXZp0H_yFoKs
2018-02-27 08:35:38,178:DEBUG:certbot.plugins.webroot:All challenges cleaned up, removing /opt/www/.well-known/acme-challenge
2018-02-27 08:35:38,178:ERROR:certbot.log:Exiting abnormally:
Traceback (most recent call last):
File "/usr/local/bin/certbot", line 11, in <module>
load_entry_point('certbot', 'console_scripts', 'certbot')()
File "/opt/certbot/src/certbot/main.py", line 861, in main
return config.func(config, plugins)
File "/opt/certbot/src/certbot/main.py", line 786, in certonly
lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
File "/opt/certbot/src/certbot/main.py", line 85, in _get_and_save_cert
lineage = le_client.obtain_and_enroll_certificate(domains, certname)
File "/opt/certbot/src/certbot/client.py", line 357, in obtain_and_enroll_certificate
certr, chain, key, _ = self.obtain_certificate(domains)
File "/opt/certbot/src/certbot/client.py", line 318, in obtain_certificate
self.config.allow_subset_of_names)
File "/opt/certbot/src/certbot/auth_handler.py", line 81, in get_authorizations
self._respond(resp, best_effort)
File "/opt/certbot/src/certbot/auth_handler.py", line 138, in _respond
self._poll_challenges(chall_update, best_effort)
File "/opt/certbot/src/certbot/auth_handler.py", line 202, in _poll_challenges
raise errors.FailedChallenges(all_failed_achalls)
FailedChallenges: Failed authorization procedure. example.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://example.com/.well-known/acme-cha
llenge/uyJ3XliXAfwmzjFhmcmc_SLHR2t7c_TH0Nbxik1TU3k: "<html>M
<head><title>404 Not Found</title></head>M
<body bgcolor="white">M
<center><h1>404 Not Found</h1></center>M
<hr><center>", www.example.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://www.example.com/.well-known/acme-challenge/moy16SD5sYeMo_WjP2EV
nIW93b5T9oVPXZp0H_yFoKs: "<html>M
<head><title>404 Not Found</title></head>M
<body bgcolor="white">M
<center><h1>404 Not Found</h1></center>M
<hr><center>"
I can.. at least parts of it :-)
version: '3.3'
services:
# Service that listens to service changes in the swam
# and notifies the reverse proxy
swarm-listener:
networks:
- proxy
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
# proxy-le acts as a man-in-the-middle between swarm-listener and docker-flow-proxy
# however, unless we're in prod, the let's encrypt setup cannot answer http challenges
# (as the pr env doesn't run on port 80 and 443)
# so we configure this with a few env variables..
# It is one of:
#- "DF_NOTIF_CREATE_SERVICE_URL=http://${STACK_NAME}_proxy:8080/v1/docker-flow-proxy/reconfigure"
#- "DF_NOTIFY_CREATE_SERVICE_URL=http://${STACK_NAME}_proxy-le:8080/v1/docker-flow-proxy-letsencrypt/reconfigure"
- "DF_NOTIFY_CREATE_SERVICE_URL=${DF_NOTIFY_CREATE_SERVICE_URL}"
- "DF_NOTIF_REMOVE_SERVICE_URL=http://${STACK_NAME}_proxy:8080/v1/docker-flow-proxy/remove"
- "DF_NOTIFY_LABEL=com.df.notify${STACK_NAME}"
image: vfarcic/docker-flow-swarm-listener:17.11.11-23
deploy:
placement:
constraints:
- node.role == manager
# The reverse proxy itself. This is HAProxy + extras.
proxy:
networks:
- proxy
environment:
- "MODE=swarm"
- "LISTENER_ADDRESS=${STACK_NAME}_swarm-listener"
- "SERVICE_NAME=${STACK_NAME}_proxy"
ports:
- "${PROXY_INGRESS_PORT_HTTP}:80"
- "${PROXY_INGRESS_PORT_HTTPS}:443"
image: vfarcic/docker-flow-proxy:17.11.17-64
deploy:
mode: global
# Only for documentation .. no effect.
depends_on:
- swarm-listener
# A Let's Encrypt helper proxy
proxy-le:
image: nib0r/docker-flow-proxy-letsencrypt
networks:
- proxy
environment:
- DF_PROXY_SERVICE_NAME=${STACK_NAME}_proxy
#- DF_SWARM_LISTENER_SERVICE_NAME=${STACK_NAME}_swarm-listener
# - LOG=debug
# - CERTBOT_OPTIONS=--staging
volumes:
# link docker socket to activate secrets support.
- /var/run/docker.sock:/var/run/docker.sock
# create a dedicated volume for letsencrypt folder.
# MANDATORY to keep persistent certificates on DFPLE.
# Without this volume, certificates will be regenerated every time DFPLE is recreated.
# OPTIONALY you will be able to link this volume to another service that also needs certificates (gitlab/gitlab-ce for example)
- le-certs:/etc/letsencrypt
deploy:
replicas: 1
labels:
- com.df.notify=true
- com.df.distribute=true
- com.df.servicePath=/.well-known/acme-challenge
- com.df.port=8080
# Our backend
app-backend:
..
training-stuff:
..
loader-thing:
..
magic-thing:
..
another-magic-thing:
..
# The database. This is exposed on its own network, only reachable
# from the app-backend
db:
..
dbbackup:
..
dbbackup-secondary:
..
some-docs-thing:
..
web-thing:
..
configs:
..
networks:
# The network for the reverse proxy (HAProxy) and associated services
proxy:
external: true
app:
driver: overlay
volumes:
..
le-certs:
Note that I'm not using com.df.notify=true in order differentiate between different environments in the same swarm cluster.
Ehm.. let me retry with the correct com.df.notify${STACK}=true label..
I can get it to work, but it's quite flaky. DNS resolution while deploying a docker stack is not stable - it must be assumed that it takes up to 1 minute for the DFP to be available. In that time-frame the code will plow through the rate limits of let's encrypt.
If you're not using com.df.notify=true doesn't that mean your container is not being proxied to?
