Cannot use manually managed Letsencrypt certificates

[houserockr]

Steps To Reproduce

I'm running a dockerized installation of bitwarden behind a reverse proxy, so I cannot have bitwarden trying to expose ports 80 or 443 to renew certificates. My reverse proxy already has its own sidecar for LE renewal.

Unfortunately, whatever value is assigned to ssl_managed_lets_encrypt in bwdata/config.yaml, I get the following error when trying to ./bitwarden.sh restart:

Status: Image is up to date for certbot/certbot:latest
docker.io/certbot/certbot:latest
docker: Error response from daemon: driver failed programming external connectivity on endpoint certbot (b1933f9860e5dbd67bd1720db8ba0d696500cfc96b9e903442cfd9045be78724): Bind for 0.0.0.0:443 failed: port is already allocated.

All my certs are located under the usual location on the host /etc/letsencrypt/live/<domain>/*.pem

I do know that bwdata/letsencrypt is mapped to /etc/letsencrypt to docker containers (via bwdata/docker/docker-compose.yml).

It seems to me that whenever there's content (files, symlinks, doesn't matter) within bwdata/letsencrypt, bitwarden starts certbot, no matter what ssl_managed_lets_encrypt is set to.

Is there any workaround to map manually managed LE certs from the host to the containers?

Here are my versions:

bitwarden.sh version 2023.4.3
Docker version 23.0.6, build ef23cbc
Docker Compose version v2.9.0

Expected Result

The config parameter ssl_managed_lets_encrypt: false should completely disable any certbot services part of bitwarden.

Actual Result

Bitwarden cannot be started due to an already used port.

Status: Image is up to date for certbot/certbot:latest
docker.io/certbot/certbot:latest
docker: Error response from daemon: driver failed programming external connectivity on endpoint certbot (b1933f9860e5dbd67bd1720db8ba0d696500cfc96b9e903442cfd9045be78724): Bind for 0.0.0.0:443 failed: port is already allocated.

Screenshots or Videos

No response

Additional Context

No response

Build Version

2023.4.3

Environment

Self-Hosted

Environment Details

• Host OS Debian 12
• Docker 23.0.6
• Docker compose v2.9.0

Issue Tracking Info

• [ ] I understand that work is tracked outside of Github. A PR will be linked to this issue should one be opened to address it, but Bitwarden doesn't use fields like "assigned", "milestone", or "project" to track progress.

[atjbramley]

Hi @houserockr,

Thank you for your report!

I was able to reproduce this issue, and I have flagged this to our engineering team.

In the meantime, if you wish to stop Bitwarden from auto-renewing LetsEncrypt certificates, you can do so by deleting or renewing the appropriate directory - please see the note in the guide below:

https://community.bitwarden.com/t/guide-to-setting-up-bitwarden-behind-an-nginx-reverse-proxy-configuration/53196#generate-and-configure-self-signed-certificate-17

If you wish to add any further information/screenshots/recordings etc., please feel free to do so at any time - our engineering team will be happy to review these.

Thanks once again!

[DrN0pe]

Had the exact same a while back, was going to automate the renewal of certs with a playbook instead that took the bit warden containers down - thanks for submitting the issue @houserockr.

[houserockr]

It's been almost a year and this issue still exists. This must be a pain for everyone running bitwarden behind a reverse proxy or alongside already running webservers.

Today I started to dig a little deeper and found a workaround and a possible fix I might even create a merge request for.

In bwdata/scripts/run.sh line 285 change the function restart() from

function restart() {
    dockerComposeDown
    dockerComposePull
    updateLetsEncrypt
    dockerComposeUp
    printEnvironment
}

to

function restart() {
    dockerComposeDown
    dockerComposePull
    # updateLetsEncrypt
    dockerComposeUp
    printEnvironment
}

After that ./bitwarden.sh start won't start the certbot container. However, note that this will only persist until you call update the next time and there's a new version of the wrapper script. So you will have to change it every time.

Here's my suggestion for a "fix":

function restart() {
    SSL_MANAGED=$(grep 'ssl_managed_lets_encrypt:' $OUTPUT_DIR/config.yml | awk '{ print $2}')
    dockerComposeDown
    dockerComposePull
    if [ "$SSL_MANAGED" = "false" ]
    then
        updateLetsEncrypt
    fi
    dockerComposeUp
    printEnvironment
}

The greps and awks to read config parameters are already used in multiple places in bwdata/scripts/run.sh

Only thing I still don't really get is the semantics of ssl_managed_lets_encrypt. Does it mean

• ssl_managed_lets_encrypt: true => bitwarden is supposed to manage the certs and
• ssl_managed_lets_encrypt: false => bitwarden is supposed to stay away from certs?

[0x4448]

https://github.com/bitwarden/self-host/blob/93fdabd92ab599b3764c6919ec1ce551942013c1/run.sh#L167C1-L168C47

The updateLetsEncrypt function only runs if the bwdata/letsencrypt/live directory exists. If your certs are managed elsewhere, removing this directory should fix it.

[houserockr]

https://github.com/bitwarden/self-host/blob/93fdabd92ab599b3764c6919ec1ce551942013c1/run.sh#L167C1-L168C47

The updateLetsEncrypt function only runs if the bwdata/letsencrypt/live directory exists. If your certs are managed elsewhere, removing this directory should fix it.

Thanks I saw that and also tried to "remove" it, but here's the problem with that: I just link /etc/letsencrypt from my host os to ./bwdata/letsencrypt. Initially I tried to link just the individual *.pem certificate files and keys into ./bwdata, but then other stuff couldn't be found.

[0x4448]

bwdata/letsencrypt is really meant for Bitwarden's managed Let's Encrypt certificates. If you use that folder for your own LE certificates, there could be unexpected results.

I use bwdata/ssl/bitwarden.example.com to store the .pem files as suggested in the documentation. Certbot is configured to use a renew hook (certbot reconfigure --cert-name bitwarden.example.com --renew-hook /path/to/renew-hook.sh) to copy files after a renewal:

#!/bin/bash
FQDN="bitwarden.example.com"

mkdir -p /opt/bitwarden/bwdata/ssl/$FQDN
cp /etc/letsencrypt/live/$FQDN/*.pem /opt/bitwarden/bwdata/ssl/$FQDN
chown -R bitwarden:bitwarden /opt/bitwarden/bwdata/ssl/$FQDN
su bitwarden -c "cd /opt/bitwarden; ./bitwarden.sh rebuild; ./bitwarden.sh start"

bwdata/config.yml:

ssl_certificate_path: /etc/ssl/bitwarden.example.com/fullchain.pem
ssl_key_path: /etc/ssl/bitwarden.example.com/privkey.pem
ssl_ca_path: /etc/ssl/bitwarden.example.com/chain.pem

[houserockr]

bwdata/letsencrypt is really meant for Bitwarden's managed Let's Encrypt certificates. If you use that folder for your own LE certificates, there could be unexpected results.

I use bwdata/ssl/bitwarden.example.com to store the .pem files as suggested in the documentation. Certbot is configured to use a renew hook (certbot reconfigure --cert-name bitwarden.example.com --renew-hook /path/to/renew-hook.sh) to copy files after a renewal:

#!/bin/bash
FQDN="bitwarden.example.com"

mkdir -p /opt/bitwarden/bwdata/ssl/$FQDN
cp /etc/letsencrypt/live/$FQDN/*.pem /opt/bitwarden/bwdata/ssl/$FQDN
chown -R bitwarden:bitwarden /opt/bitwarden/bwdata/ssl/$FQDN
su bitwarden -c "cd /opt/bitwarden; ./bitwarden.sh rebuild; ./bitwarden.sh start"

bwdata/config.yml:

ssl_certificate_path: /etc/ssl/bitwarden.example.com/fullchain.pem
ssl_key_path: /etc/ssl/bitwarden.example.com/privkey.pem
ssl_ca_path: /etc/ssl/bitwarden.example.com/chain.pem

I read that the bwdata/ssl folder is mapped into the containers as well and I did indeed try this. But that must have been months ago and it didn't work either back then. I'll give it another shot. Thank you!

Question tho, because you are copying the certs after every renewal, do you think it wouid generally work with symlinks per cert file?

[houserockr]

I tried it again and the solution provided by @0x4448 and the official documentation works as expected. Important to note is to set ssl_managed_lets_encrypt: false and the parameters ssl_* in bwdata/config.yml as shown in the solution