dokku-letsencrypt
dokku-letsencrypt copied to clipboard
dokku
Error renewing certificates with different proxy port configuration
I am having difficulty auto-renewing Dokku LetsEncrypt certificates. In our configuration, we removed the default proxy ports and added a mapping for port 443 to our backend server running on 8000:
dokku proxy:ports-add caregiving-app https:443:8000
I am getting errors when trying to renew the certificate with the proxy port mapping configuration:
error: one or more domains had a problem:
[caregiving.geri.life] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: 65.21.181.240: Invalid response from http://.../.well-known/acme-challenge/P7MiJYiHg3k6rPjnukE5Gi1ATShFk5txYUF98cOaUSc: 404
Originally posted by @brylie in https://github.com/dokku/dokku-letsencrypt/issues/264#issuecomment-1191546667
I rebuilt the caregiving-app and changed the Docker file to use the Dokku default port of 5000. However, here is a similar app that is failing to renew its Lets Encrypt certificate:
dokku report companionship-care-app
# dokku report companionship-care-app
-----> uname: Linux beta 5.4.0-107-generic #121-Ubuntu SMP Thu Mar 24 16:04:27 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
-----> memory:
total used free shared buff/cache available
Mem: 1935 818 96 28 1020 900
Swap: 0 0 0
-----> docker version:
Client: Docker Engine - Community
Version: 20.10.17
API version: 1.41
Go version: go1.17.11
Git commit: 100c701
Built: Mon Jun 6 23:02:57 2022
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.17
API version: 1.41 (minimum version 1.12)
Go version: go1.17.11
Git commit: a89b842
Built: Mon Jun 6 23:01:03 2022
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.6
GitCommit: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
runc:
Version: 1.1.2
GitCommit: v1.1.2-0-ga916309
docker-init:
Version: 0.19.0
GitCommit: de40ad0
-----> docker daemon info:
WARNING: No swap limit support
Client:
Context: default
Debug Mode: true
Plugins:
app: Docker App (Docker Inc., v0.9.1-beta3)
buildx: Docker Buildx (Docker Inc., v0.8.2-docker)
scan: Docker Scan (Docker Inc., v0.17.0)
Server:
Containers: 14
Running: 5
Paused: 0
Stopped: 9
Images: 109
Server Version: 20.10.17
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
runc version: v1.1.2-0-ga916309
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 5.4.0-107-generic
Operating System: Ubuntu 20.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 1.89GiB
Name: beta
ID: BQ67:2GLT:TCOT:EZUQ:RRP5:QJAM:RMC6:LPNG:NOS4:CHAK:T4PW:RJXU
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
-----> git version: git version 2.25.1
-----> sigil version: 0.9.0build+bc921b7
-----> herokuish version:
herokuish: 0.5.36
buildpacks:
heroku-buildpack-multi v1.2.0
heroku-buildpack-ruby v240
heroku-buildpack-nodejs v196
heroku-buildpack-clojure v87
heroku-buildpack-python v211
heroku-buildpack-java v70
heroku-buildpack-gradle v36
heroku-buildpack-scala v92
heroku-buildpack-play v26
heroku-buildpack-php v218
heroku-buildpack-go v162
heroku-buildpack-nginx v16
buildpack-null v3
-----> dokku version: dokku version 0.27.8
-----> plugn version: plugn: 0.12.0build+3a27594
-----> dokku plugins:
00_dokku-standard 0.27.8 enabled dokku core standard plugin
20_events 0.27.8 enabled dokku core events logging plugin
app-json 0.27.8 enabled dokku core app-json plugin
apps 0.27.8 enabled dokku core apps plugin
builder 0.27.8 enabled dokku core builder plugin
builder-dockerfile 0.27.8 enabled dokku core builder-dockerfile plugin
builder-herokuish 0.27.8 enabled dokku core builder-herokuish plugin
builder-null 0.27.8 enabled dokku core builder-null plugin
builder-pack 0.27.8 enabled dokku core builder-pack plugin
buildpacks 0.27.8 enabled dokku core buildpacks plugin
certs 0.27.8 enabled dokku core certificate management plugin
checks 0.27.8 enabled dokku core checks plugin
common 0.27.8 enabled dokku core common plugin
config 0.27.8 enabled dokku core config plugin
cron 0.27.8 enabled dokku core cron plugin
docker-options 0.27.8 enabled dokku core docker-options plugin
domains 0.27.8 enabled dokku core domains plugin
enter 0.27.8 enabled dokku core enter plugin
git 0.27.8 enabled dokku core git plugin
letsencrypt 0.18.1 enabled Automated installation of let's encrypt TLS certificates
logs 0.27.8 enabled dokku core logs plugin
network 0.27.8 enabled dokku core network plugin
nginx-vhosts 0.27.8 enabled dokku core nginx-vhosts plugin
plugin 0.27.8 enabled dokku core plugin plugin
postgres 1.18.0 enabled dokku postgres service plugin
proxy 0.27.8 enabled dokku core proxy plugin
ps 0.27.8 enabled dokku core ps plugin
registry 0.27.8 enabled dokku core registry plugin
repo 0.27.8 enabled dokku core repo plugin
resource 0.27.8 enabled dokku core resource plugin
run 0.27.8 enabled dokku core run plugin
scheduler 0.27.8 enabled dokku core scheduler plugin
scheduler-docker-local 0.27.8 enabled dokku core scheduler-docker-local plugin
scheduler-null 0.27.8 enabled dokku core scheduler-null plugin
shell 0.27.8 enabled dokku core shell plugin
ssh-keys 0.27.8 enabled dokku core ssh-keys plugin
storage 0.27.8 enabled dokku core storage plugin
trace 0.27.8 enabled dokku core trace plugin
=====> companionship-care-app app-json information
App json computed selected: app.json
App json global selected: app.json
App json selected:
=====> companionship-care-app app information
App created at: 1648919308
App deploy source: companionship-care-app
App deploy source metadata: companionship-care-app
App dir: /home/dokku/companionship-care-app
App locked: false
=====> companionship-care-app builder information
Builder build dir:
Builder computed build dir:
Builder computed selected:
Builder global build dir:
Builder global selected:
Builder selected:
=====> companionship-care-app builder-dockerfile information
Builder dockerfile computed dockerfile path: Dockerfile
Builder dockerfile global dockerfile path: Dockerfile
Builder dockerfile dockerfile path:
=====> companionship-care-app builder-pack information
Builder pack computed projecttoml path: project.toml
Builder pack global projecttoml path: project.toml
Builder pack projecttoml path:
=====> companionship-care-app buildpacks information
Buildpacks computed stack: gliderlabs/herokuish:latest-20
Buildpacks global stack:
Buildpacks list:
Buildpacks stack:
=====> companionship-care-app ssl information
Ssl dir: /home/dokku/companionship-care-app/tls
Ssl enabled: true
Ssl hostnames: beta.companionship.care
Ssl expires at: Jul 1 15:29:09 2022 GMT
Ssl issuer: C = US, O = Let's Encrypt, CN = R3
Ssl starts at: Apr 2 15:29:10 2022 GMT
Ssl subject: subject=CN = beta.companionship.care
Ssl verified: self signed
=====> companionship-care-app checks information
Checks disabled list: none
Checks skipped list: none
=====> companionship-care-app cron information
Cron task count: 0
=====> companionship-care-app docker options information
Docker options build: --link dokku.postgres.companionship-care-db:dokku-postgres-companionship-care-db
Docker options deploy: --link dokku.postgres.companionship-care-db:dokku-postgres-companionship-care-db --restart=on-failure:10 -v /var/lib/dokku/data/storage/companionship-care-app:/app/project/media
Docker options run: --link dokku.postgres.companionship-care-db:dokku-postgres-companionship-care-db -v /var/lib/dokku/data/storage/companionship-care-app:/app/project/media
=====> companionship-care-app domains information
Domains app enabled: true
Domains app vhosts: beta.companionship.care
Domains global enabled: true
Domains global vhosts: beta.companionship.care
=====> companionship-care-app git information
Git deploy branch: master
Git global deploy branch: main
Git keep git dir: false
Git rev env var: GIT_REV
Git sha:
Git last updated at: 1648918513
=====> companionship-care-app letsencrypt information
Letsencrypt active: true
Letsencrypt autorenew: true
Letsencrypt email: [email protected]
Letsencrypt expiration: 1656689349
=====> companionship-care-app logs information
Logs computed max size: 10m
Logs global max size: 10m
Logs global vector sink:
Logs max size:
Logs vector sink:
=====> companionship-care-app network information
Network attach post create:
Network attach post deploy:
Network bind all interfaces: false
Network computed attach post create:
Network computed attach post deploy:
Network computed bind all interfaces: false
Network computed initial network:
Network computed tld:
Network global attach post create:
Network global attach post deploy:
Network global bind all interfaces: false
Network global initial network:
Network global tld:
Network initial network:
Network static web listener:
Network tld:
Network web listeners: 172.17.0.3:8000
=====> companionship-care-app nginx information
Nginx access log format:
Nginx access log path: /var/log/nginx/companionship-care-app-access.log
Nginx bind address ipv4:
Nginx bind address ipv6: ::
Nginx client max body size: 10m
Nginx disable custom config: false
Nginx error log path: /var/log/nginx/companionship-care-app-error.log
Nginx global hsts: true
Nginx computed hsts: true
Nginx hsts:
Nginx hsts include subdomains: true
Nginx hsts max age: 15724800
Nginx hsts preload: false
Nginx proxy buffer size: 4096
Nginx proxy buffering: on
Nginx proxy buffers: 8 4096
Nginx proxy busy buffers size: 8192
Nginx proxy read timeout: 60s
Nginx last visited at: 1658562818
Nginx x forwarded for value: $remote_addr
Nginx x forwarded port value: $server_port
Nginx x forwarded proto value: $scheme
Nginx x forwarded ssl:
=====> companionship-care-app proxy information
Proxy enabled: true
Proxy port map: http:8000:8000 https:443:8000
Proxy type: nginx
=====> companionship-care-app ps information
Deployed: true
Processes: 1
Ps can scale: true
Ps computed procfile path: Procfile
Ps global procfile path: Procfile
Ps procfile path:
Ps restart policy: on-failure:10
Restore: true
Running: true
Status web 1: running (CID: 427cf84c048)
=====> companionship-care-app registry information
Registry computed image repo: dokku/companionship-care-app
Registry computed push on release: false
Registry computed server:
Registry global push on release:
Registry global server:
Registry image repo:
Registry push on release:
Registry server:
Registry tag version:
=====> companionship-care-app resource information
=====> companionship-care-app scheduler information
Scheduler computed selected: docker-local
Scheduler global selected: docker-local
Scheduler selected:
=====> companionship-care-app scheduler-docker-local information
Scheduler docker local disable chown:
Scheduler docker local parallel schedule count:
=====> companionship-care-app storage information
Storage build mounts:
Storage deploy mounts: -v /var/lib/dokku/data/storage/companionship-care-app:/app/project/media
Storage run mounts: -v /var/lib/dokku/data/storage/companionship-care-app:/app/project/media
Error log for: dokku letsencrypt:auto-renew companionship-care-app
# dokku letsencrypt:auto-renew companionship-care-app
companionship-care-app
=====> Auto-renew companionship-care-app...
=====> Enabling letsencrypt for companionship-care-app
-----> Enabling ACME proxy for companionship-care-app...
Reloading nginx configuration (via systemctl): nginx.service.
-----> Getting letsencrypt certificate for companionship-care-app...
- Domain 'beta.companionship.care'
2022/07/23 08:33:45 [INFO] [beta.companionship.care] acme: Obtaining bundled SAN certificate
2022/07/23 08:33:46 [INFO] [beta.companionship.care] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/133224408626
2022/07/23 08:33:46 [INFO] [beta.companionship.care] acme: Could not find solver for: tls-alpn-01
2022/07/23 08:33:46 [INFO] [beta.companionship.care] acme: use http-01 solver
2022/07/23 08:33:46 [INFO] [beta.companionship.care] acme: Trying to solve HTTP-01
2022/07/23 08:33:52 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/133224408626
2022/07/23 08:33:52 Could not obtain certificates:
error: one or more domains had a problem:
[beta.companionship.care] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: 65.21.181.240: Invalid response from http://beta.companionship.care/.well-known/acme-challenge/6539zpMCJtvBFZFr_9HDqkO5HTtyZJyaB3q9bOvbKIM: 404
-----> Certificate retrieval failed!
-----> Disabling ACME proxy for companionship-care-app...
Reloading nginx configuration (via systemctl): nginx.service.
! Failed to setup letsencrypt
! Check log output for further information on failure
Hey, any news on that, i have the same problem with a fresh install of dokku. The first certificate for the first app works, the second one produces exactly the same error message that you got. I tried it already twice by installing dokku completly again on a fresh server.
I think you're going to need to add a proxy mapping for port 80. As you see in the http-01 challenge, it says its requesting on http, meaning that you need to have an nginx config that listens on that port.