Unable to use SSL with Cloudflare Origin Certificates along with kamal-proxy

[nathanpalmer]

I've tried several different ways to make this work with out of the box options and I don't see that there is currently a way. I thought I would document here what I'm trying to do and get any feedback from the team on it.

The infrastructure looks essentially like this.

┌────────────────┐
│   Cloudflare   │
└────────────────┘
    │        │
┌───────┐┌───────┐
│ Web 1 ││ Web 2 │
└───────┘└───────┘

Public SSL is going to be handled by Cloudflare. However, I do need to install Cloudflare Origin Certificates so that communication between the Cloudflare server and the backend Web servers are encrypted.

It seems the only supported configuration for kamal at this point is to expose a HTTP interface or HTTPS using automatic certificate management. It's possible the work on #969 is going to allow this to be a configuration though. That might simply be the eventual solution, but I can't tell that there is any movement on it. (It also seems like maybe we're waiting on volume/file mounting work from @djmb)

For now I've hacked together a messy solution that includes:

1. Fork of kamal to hardcode the tls-certificate-path and tls-private-key-path and disable the ensure_one_host_for_ssl check.
2. Adding a pre-deploy script that pulls the certificate information from 1password and creates the cert.pem and key.pem
3. In the pre-deploy script also create a .kamal/proxy/options file that mounts the folder with the certificates and also adds the publish ports that are in the default options
4. Run kamal setup
5. Run kamal proxy reboot (because the configuration changed on deploy and we have to restart it.)

I also tried to by-pass kamal-proxy altogether and configure puma with the certs, which worked, but redeploys are failing because of port assignments. More info in #1133.

I could streamline by hack if we had a hook that ran pre-proxy-deploy so I could copy the certs over in that step.. as well as a way to add arbitrary options to the proxy command.

But also open to more direct solutions for this specific setup.

[gcrofils]

As a follow-up and after a few hours of trial and error, here’s my KISS workaround to get wildcard Cloudflare certs working. Since this is a temporary setup, I chose to hardcode paths and manually copy certs on the target server. Simpler than tweaking scripts for now. My goal was to migrate a multitenant service from Heroku.

Hope this helps while the long-term solution gets merged.

Disclaimer:

• This is a temporary, hardcoded setup
• Manual cert installation on a single target server
• certs are shared by all applications on that server.

---

1. Forked Kamal and modified Kamal::Configuration::Proxy#deploy_options:

https://github.com/basecamp/kamal/blob/8fe2f921645ad8416fabd2aae64a0ca2000487ad/lib/kamal/configuration/proxy.rb#L29

def deploy_options
  {
    ...
    "log-response-header": proxy_config.dig("logging", "response_headers"),
    "tls-certificate-path": "/home/kamal-proxy/.config/certs/cert.pem",
    "tls-private-key-path": "/home/kamal-proxy/.config/certs/key.pem",
  }.compact
end

In Gemfile:

gem 'kamal', require: false, git: "https://github.com/USERNAME/kamal"

2. Manually copy the certs to the server:

/etc/kamal/certs/
├── cert.pem  (644)
└── key.pem   (644)

3. Add .kamal/proxy/options on the server to add a volume:

--publish 80:80 --publish 443:443 --log-opt max-size=10m --volume /etc/kamal/certs:/home/kamal-proxy/.config/certs:ro 

This works because kamal reads this file when launching the proxy container:

https://github.com/basecamp/kamal/blob/8fe2f921645ad8416fabd2aae64a0ca2000487ad/test/commands/proxy_test.rb#L18

4. kamal deploy config

proxy:
  ssl: true
  host: example.tld,*.example.tld
  forward_headers: true

[gcrofils]

As a follow up of my previous comment, I decided to use this PR in production. https://github.com/basecamp/kamal/pull/1531#issuecomment-2958709501

More convenient that the hack I described above.

[djmb]

Closed by https://github.com/basecamp/kamal/pull/1531