feat: Allow local certificates to be used by Caddy

[thePanz]

Relates to #79 and #93

[B-Galati]

Great! Thank you @thePanz :pray:

[dunglas]

Awesome! As it's a relatively specific use case, I would prefer to not add too much "unnecessary" comments to the skeleton itself. Could we just add a doc entry explaining what lines to add as we've done for XDebug? We can use the ```patch Markdown trick for this purpose.

[maxhelias]

Wow! Thanks @thePanz 🎉

@dunglas, maybe rework the xdebug part to have only one page on development options (xdebug + tls) ?

[dunglas]

@maxhelias, I'm not sure. I think having small pages with descriptive URLs and titles will improve SEO.

[zmitic]

After many hours, I failed to make it work. My setup is Linux mint, and all I get is non-responsive page.

Troubleshooting page with docker cp command also doesn't work, even with changed path (usr/share instead of usr/local/share).

I am out of ideas.

UPDATE:

Because caddy.community forums are down for a long time, I still couldn't find a solution.

This is my config:

CaddyFile

{
    # Debug
    {$DEBUG}
    # HTTP/3 support
    servers {
        protocol {
            experimental_http3
        }
    }
}

{$SERVER_NAME}

{$CADDY_TLS_CONFIG}

log

route {
    root * /srv/app/public
    mercure {
        # Transport to use (default to Bolt)
        transport_url {$MERCURE_TRANSPORT_URL:bolt:///data/mercure.db}
        # Publisher JWT key
        publisher_jwt {env.MERCURE_PUBLISHER_JWT_KEY} {env.MERCURE_PUBLISHER_JWT_ALG}
        # Subscriber JWT key
        subscriber_jwt {env.MERCURE_SUBSCRIBER_JWT_KEY} {env.MERCURE_SUBSCRIBER_JWT_ALG}
        # Allow anonymous subscribers (double-check that it's what you want)
        anonymous
        # Enable the subscription API (double-check that it's what you want)
        subscriptions
        # Extra directives
        {$MERCURE_EXTRA_DIRECTIVES}
    }
    vulcain
    push
    php_fastcgi unix//var/run/php/php-fpm.sock
    encode zstd gzip
    file_server
}

And

docker-compose.override.yml

version: "3.4"

# Development environment override
services:
  php:
    volumes:
      # The "cached" option has no effect on Linux but improves performance on Mac
      - ./:/srv/app:rw,cached
      - ./docker/php/conf.d/symfony.dev.ini:/usr/local/etc/php/conf.d/symfony.ini
      # If you develop on Linux, comment out the following volumes to just use bind-mounted project directory from host
      # - ./var:/srv/app/var:rw
      # If you develop on Mac you can remove the var/ directory from the bind-mount
      # for better performance by enabling the next line 
      # - /srv/app/var
    environment:
      APP_ENV: dev

  caddy:

    environment: 
      # Comment out the following line to use the local TLS certificates
      CADDY_TLS_CONFIG: "tls /etc/caddy/certs/tls.pem /etc/caddy/certs/tls.key"
    volumes:
      # Comment out the following volume enable the caddy/certs volume, needed when CADDY_TLS_CONFIG is in use
      - ./docker/caddy/certs:/etc/caddy/certs:ro
      - ./docker/caddy/Caddyfile:/etc/caddy/Caddyfile:ro
      - ./public:/srv/app/public:ro
###> doctrine/doctrine-bundle ###
  database:
    ports:
      - "5432"
###< doctrine/doctrine-bundle ###

---

I run mkcert -cert-file docker/caddy/certs/tls.pem -key-file docker/caddy/certs/tls.key "api.localhost" and built it like:

SERVER_NAME="api.localhost, caddy:80" docker-compose up -d --build

Container restarted, built again... doesn't matter. Not even if I build it without config and restarting it later; everytime logs are the same:

{
  "level": "info",
  "ts": 1627129136.5092123,
  "msg": "using provided configuration",
  "config_file": "/etc/caddy/Caddyfile",
  "config_adapter": "caddyfile",
}

run: adapting config using caddyfile: 
server listening on [:80] is HTTP, but attempts to configure TLS connection policies

---

Restarting browser or running sudo update-ca-certificates (in the act of desperation) didn't help; all I get is ERR_CONNECTION_REFUSED

[mario-fehr]

@zmitic the problem is the caddy:80 in SERVER_NAME="api.localhost, caddy:80". It is not possible to configure tls on sites that listen on the HTTP port.

@thePanz @maxhelias @dunglas any suggestions to solve this issue?

Edit: this issue is also present in de default build SERVER_NAME: ${SERVER_NAME:-localhost, caddy:80} of the dockerfile.

[Jayfrown]

Thanks, I got it to work as follows:

Expand diff

❯ git diff --staged | cat        
diff --git a/docker-compose.override.yml b/docker-compose.override.yml
index 0143fec..2ab2b3d 100644
--- a/docker-compose.override.yml
+++ b/docker-compose.override.yml
@@ -17,6 +17,10 @@ services:
     volumes:
       - ./docker/caddy/Caddyfile:/etc/caddy/Caddyfile:ro
       - ./public:/srv/app/public:ro
+      - ./docker/caddy/certs:/etc/caddy/certs:ro
+    environment:
+      SERVER_NAME: ${SERVER_NAME:-localhost, caddy}
+      CADDY_TLS_CONFIG: "tls /etc/caddy/certs/tls.pem /etc/caddy/certs/tls.key"
 
   phpldapadmin:
     image: osixia/phpldapadmin:latest
diff --git a/docker-compose.yml b/docker-compose.yml
index c1a1ea9..044033f 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -20,7 +20,7 @@ services:
       start_period: 30s
     environment:
       DATABASE_URL: "mysqli://root:${MYSQL_ROOT_PASSWORD:-!ChangeMe!}@database:3306/${MYSQL_DATABASE:-app}?serverVersion=mariadb-10.3.13"
-      MERCURE_URL: ${CADDY_MERCURE_URL:-http://caddy/.well-known/mercure}
+      MERCURE_URL: ${CADDY_MERCURE_URL:-https://caddy/.well-known/mercure}
       MERCURE_PUBLIC_URL: https://${SERVER_NAME:-localhost}/.well-known/mercure
       MERCURE_JWT_SECRET: ${CADDY_MERCURE_JWT_SECRET:-!ChangeMe!}
       JWT_PASSPHRASE: ${JWT_PASSPHRASE:-!ChangeMe!}
diff --git a/docker/caddy/Caddyfile b/docker/caddy/Caddyfile
index 6d5387c..18f70c3 100644
--- a/docker/caddy/Caddyfile
+++ b/docker/caddy/Caddyfile
@@ -11,6 +11,8 @@
 
 {$SERVER_NAME}
 
+{$CADDY_TLS_CONFIG}
+
 log
 
 route {
diff --git a/docker/caddy/certs/.gitignore b/docker/caddy/certs/.gitignore
new file mode 100644
index 0000000..65e3d29
--- /dev/null
+++ b/docker/caddy/certs/.gitignore
@@ -0,0 +1,3 @@
+# Ignore locally generated certs for Caddy
+tls.key
+tls.pem

I added the .gitignore because in a multi-developer project you would want everyone to have their own locally generated CA and certs.

---

Now caddy uses the locally generated certificates, which means the cert is trusted within my browser. However, I found two issues:

1. The CA generated by mkcert is not automatically trusted within the docker PHP container, meaning it cannot send Mercure updates:

❯ docker compose exec php /bin/sh
/srv/app # ./bin/console hautelook:fixtures:load
Careful, database will be purged. Do you want to continue y/N ?y

In Hub.php line 104:
  Failed to send an update.  

In CommonResponseTrait.php line 148:
  SSL certificate problem: unable to get local issuer certificate for "https://caddy/.well-known/mercure".

This renders the entire exercise rather useless.

2. The CA generated by mkcert is not automatically trusted in my local PHP set-up, so local commands that try to send Mercure updates fail in the same way.

As the CA is now (relatively) static I can add it to the trust store used by local PHP as follows:

cat "$(mkcert -CAROOT)/rootCA.pem" >> /usr/local/etc/[email protected]/cert.pem

Note that this might be a MacOS / brew PHP specific issue. Can someone confirm/refute whether the CA generated by mkcert is automatically trusted in local PHP when run on Linux?

[Jayfrown]

Both issues could be fixed if there is a separate configuration within the Caddyfile for caddy:80 and localhost:443, such that both local PHP and the docker PHP app can send updates to http://caddy/.well-known/mercure (as is the default for the dockerized PHP app). The stanza for localhost:443 could then use the certs generated by mkcert.

[mario-fehr]

Any plans to fix and merger this?

[maxhelias]

I updated this PR

[thePanz]

Thanks @maxhelias for the update, sorry for not following up on this :+1:

[maxhelias]

@dunglas what do you thing about this rework ?

[dunglas]

This looks good to me. @mholt would you mind checking if it's in sync with Caddy's best practices?