SSL Configuration and Reverse Proxy Challenges in OneUptime Docker-Compose Setup

[torstenhoegel]

Describe the bug The current OneUptime Docker-Compose setup assumes SSL is managed externally. I have configured SSL using NGINX with Certbot on the host instance. The issue arises when trying to set up the status page host. Despite configuring an A record and setting up NGINX to proxy traffic to the container, the status page fails to work correctly. The container expects SSL connections even on port 443, but the dashboard process does not handle SSL, leading to proxying issues.

The following NGINX configuration is used for the status page:

server {
    listen 443 ssl;
    server_name <redactedDomain>;

    # SSL Certificates for Nginx
    ssl_certificate /etc/letsencrypt/live/<redactedDomain>/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/<redactedDomain>/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

    location / {
        proxy_pass https://localhost:4035; # Route to OneUptime's container port
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_ssl_verify off;
    }

    # Optional: Custom error handling
    error_page 502 /502.html;
    location = /502.html {
        root /usr/share/nginx/html;
        internal;
    }
}

# Catch-All Block
server {
    listen 443 ssl default_server;
    server_name _;

    ssl_certificate /etc/letsencrypt/live/<redactedDomain>/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/<redactedDomain>/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

    location / {
        proxy_pass https://localhost:4035;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_ssl_verify off;
    }
}

server {
    listen 80 default_server;
    server_name _;

    return 301 https://$host$request_uri;
}

When the system runs, the ingress container logs the following error:

2024/11/27 15:28:36 [error] 28#28: *5685 cannot load certificate "/etc/nginx/certs/StatusPageCerts/.crt": BIO_new_file() failed (SSL: error:80000002:system library::No such file or directory:calling fopen(/etc/nginx/certs/StatusPageCerts/.crt, r) error:10000080:BIO routines::no such file) while SSL handshaking, client: 172.19.0.1, server: 0.0.0.0:7850

It appears the container expects SSL certificates to be handled internally or placed in a specific directory (/etc/nginx/certs/StatusPageCerts), but this is not documented, nor is it clear how to integrate external SSL handling with the container setup.

To Reproduce Steps to reproduce the behavior: 1. Deploy OneUptime using Docker-Compose. 2. Configure an A record to point to the instance running the status page. 3. Set up NGINX with SSL termination and proxy traffic to the container. 4. Attempt to access the status page and observe the error.

Expected behavior The status page should work seamlessly with externally terminated SSL (via NGINX). Alternatively, the container should be capable of handling SSL termination itself without requiring complex workarounds.

Screenshots N/A

Desktop • OS: Ubuntu 22.04 • Browser: not relevant • Version: not relevant

Deployment Type Self-hosted

Additional context • Documentation on SSL handling within the container and integrating with reverse proxies is unclear. • The expectation that SSL is managed externally conflicts with the container’s behavior and logs. • A clear guide for deploying with external SSL termination and reverse proxies (e.g., NGINX, Traefik) would resolve these issues.

[torstenhoegel]

@simlarsen would you mind to elaborate on this?

[simlarsen]

Looking into this. Should have an update on this soon.

[torstenhoegel]

Sounds great! Really looking forward to that

[torstenhoegel]

@simlarsen any updates on this?

[Tyler-RCSD]

I have the exact same problem described here. Any updates @simlarsen? Were you able to work around this issue @torstenhoegel?

[torstenhoegel]

@Tyler-RCSD sadly not, tbh, not having an active deployment at this time, since the resource usage of one uptime goes through the roof at times, but my only approach was to use a Kubernetes (self hosted is fine), ignoring the custom domains. This is not a good implementation to have two separate nginx's and only works in big clusters, where you would have multiple ip's being able to propagate, but for smaller deployments this just does not work.

[sevensolutions]

I know the question was about NGINX, but maybe it helps someone. This is what i've done to make it working behind a proxy. But I'am using traefik. OneUptime is deployed using docker compose as described here: https://oneuptime.com/docs/installation/docker-compose

I've changed these OneUptime settings in config.env:

HOST=oneuptime.mydomain.com

# This was also changed to free port 80 for traefik
ONEUPTIME_HTTP_PORT=8080

HTTP_PROTOCOL=https

STATUS_PAGE_HTTPS_PORT=8443

STATUS_PAGE_CNAME_RECORD=oneuptime.mydomain.com

# This was also needed because we changed ONEUPTIME_HTTP_PORT.
# Otherwise the probes were not connected.
GLOBAL_PROBE_1_ONEUPTIME_URL=http://localhost:8080
GLOBAL_PROBE_2_ONEUPTIME_URL=http://localhost:8080

My traefik config:

tls:
  certificates:
    - certFile: /etc/traefik/certs/mydomain.com.pem
      keyFile: /etc/traefik/certs/mydomain.com.key

http:
  services:
    oneuptime:
      loadBalancer:
        servers:
          - url: http://oneuptime-ingress-1:7849

  middlewares:
    redirectHttpToHttps:
      redirectScheme:
        scheme: https
        permanent: true

  routers:
    oneuptime:
      entryPoints: ["web", "websecure"]
      tls:
        domains:
          - main: "mydomain.com"
            sans:
              - "*.mydomain.com"
      rule: "Host(`oneuptime.mydomain.com`)"
      service: oneuptime
    oneuptime-http:
      entryPoints: ["web"]
      rule: "Host(`oneuptime.mydomain.com`)"
      middlewares: ["redirectHttpToHttps"]
      service: oneuptime

    status-mydomain:
      entryPoints: ["websecure"]
      tls:
        domains:
          - main: "mydomain.com"
            sans:
              - "*.mydomain.com"
      rule: "Host(`status.mydomain.com`)"
      service: oneuptime
    status-mydomain-http:
      entryPoints: ["web"]
      rule: "Host(`status.mydomain.com`)"
      middlewares: ["redirectHttpToHttps"]
      service: oneuptime


    status-another-status-page:
      entryPoints: ["web"]
      rule: "Host(`status.anotherdomain.com`)"
      #middlewares: ["redirectHttpToHttps"]
      service: oneuptime

[Tyler-RCSD]

Thanks for sharing @sevensolutions! I've tried to adapt your traefik config to nginx but I'm still stuck in a similar spot however. My config.env looks like:

HOST=status.mydomain.com
ONEUPTIME_HTTP_PORT=8080
HTTP_PROTOCOL=HTTPS
STATUS_PAGE_HTTPS_PORT=8443
STATUS_PAGE_CNAME_RECORD=status.mydomain.com

and nginx looks like this:

server {
    listen 80;
    server_name status.mydomain.com;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl;
    server_name status.mydomain.com;

    ssl_certificate /etc/SSL/mycert.crt;
    ssl_certificate_key /etc/SSL/mycert.key;

    location / {
        proxy_pass http://localhost:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

server {
    listen 80;
    server_name status.mydomain.com;

    location / {
        proxy_pass http://localhost:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

When I run npm start I get "✅App endpoint is up (localhost)" but then it sits on "App (Status Check) localhost/status returned HTTP 404" forever.

[sevensolutions]

Oh i should have said, that i'am not using npm start or npm at all to manage it. I'am running it with (export $(grep -v '^#' config.env | xargs) && docker compose up --remove-orphans -d) as noted in the mentioned docker compose guide.

I think in this case it will skip some status checks. So i dont know if they're also maybe failing in my setup but the app itself and also the status pages are working fine.

[Tyler-RCSD]

That's helped! It does seem that npm start does status checks against the application and none of them are working. If I run sudo bash -c "(export $(grep -v '^#' config.env | xargs) && docker compose up --remove-orphans -d)" as described in the docker-compose documentation things start up without doing those checks.

Things were still broken.... but then I figured out that it was SELinux, which seems to get me every time. Had to do sudo setsebool -P httpd_can_network_connect on to allow it through.

For posterity I'll also mention that my config.env remains exactly as I described above however I did modify my nginx.conf to replace http://localhost:8080 with http://status.mydomain.com:8080, using the same DNS name specified in HOST. I'm not certain whether or not this was necessary to get it going, but perhaps it'll help someone.

Thanks Daniel!

[GunniBusch]

So for anyone wanting another possible workaround without any compromises besides having 2 nginx containers.

For the second nginx container: In theory you just have to terminate ssl for you main domain eg. uptime.ex.com. Yeah... This would be the solution if you want to use a different port for the main domain, since port 443 is already used by status page.

And to handle the 443 of status page we have to proxy it too. But how can we do that without terminating ssl and still use the certs from oneuptime?

Just use NGINX SNI to reroute it. But before that we have to make nginx listen on an internal port eg 8443 and terminate ssl for the main dashboard. Then we can just ssl forward both.

So in practical terms it looks like this:

stream {
    map $ssl_preread_server_name $backend {
        uptime.ex.com backend_http;  # SSL termination
        default backend_ssl;                 # SSL passthrough
    }

    upstream backend_http {
        server 127.0.0.1:8443;  # Internal handoff to HTTP processing
    }

    upstream backend_ssl {
        server ingress:7850;  # SSL passthrough
    }

    server {
        listen 443;
        proxy_pass $backend;
        ssl_preread on;
    }
}

events {}

http {
    upstream ingress_http {
        server ingress:7849;
    }

    server {
        listen 80;
        server_name _;
        # Redirect HTTP to HTTPS
        location / {
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_pass http://ingress_http;
        }
    }
    server {
        listen 80;
        server_name uptime.ex.com;

        # Redirect HTTP to HTTPS
        return 301 https://$host$request_uri;
    } 
   
    server {
        listen 8443 ssl;  # Internal only, not exposed
        server_name uptime.ex.com;

        ssl_certificate /etc/letsencrypt/live/uptime.ex.com/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/uptime.ex.com/privkey.pem;

        location / {
            proxy_pass http://ingress_http;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto https;
        }
    }
}

the important thing is to change the ports on the Ingress container to expose:

expose:
             - 7849
             - 7850
  

[phillf]

Does anyone have a working Traefik v3 config? I am struggling a little bit with getting this working.

When I attempt to login I get the following error: user should logged in to access this API.

Here are my Traefik labels:

labels:
            # Enable Traefik
            - traefik.enable=true

            # Oneuptime - Dashboard - HTTPS/TLS - Web UI
            - traefik.http.routers.oneuptime-https.tls=true
            - traefik.http.routers.oneuptime-https.entrypoints=https
            - traefik.http.routers.oneuptime-https.rule=Host(`overwatch.creativetech.me`)
            - traefik.http.services.oneuptime-https.loadbalancer.server.port=7849
            - traefik.http.routers.oneuptime-https.service=oneuptime-https@docker

            # Oneuptime - Status Page - HTTPS/TLS - Web UI
            - traefik.http.routers.oneuptime-statuspage-https.tls=true
            - traefik.http.routers.oneuptime-statuspage-https.entrypoints=https
            - traefik.http.routers.oneuptime-statuspage-https.rule=Host(`status.creativetech.me`)
            - traefik.http.services.oneuptime-statuspage-https.loadbalancer.server.port=7850
            - traefik.http.routers.oneuptime-statuspage-https.service=oneuptime-statuspage-https@docker