[BUG] SSL Postgres: FATAL: could not load private key file

[Tmmcmasters]

I am happy to see this being released. I am running into an error still on version 4.0.0-beta.400.

FATAL: could not load private key file "/var/lib/postgresql/certs/server.key": Permission denied 2025-03-27 19:55:19.728 UTC [1] LOG: database system is shut down 2025-03-27 19:55:32.860 UTC [1] FATAL: could not load private key file "/var/lib/postgresql/certs/server.key": Permission denied 2025-03-27 19:55:32.860 UTC [1] LOG: database system is shut down

Steps are: Make publicly available on the normal port of 5432. Once it is started, I stop it, and click enable SSL: I get an empty Proxy Logs which showed the first weird thing. I have to hard refresh CTRL + SHIFT + R to get this page to work because it freezes my client. After hard refresh, I start the postgres server: FATAL: could not load private key file "/var/lib/postgresql/certs/server.key": Permission denied 2025-03-27 19:55:19.728 UTC [1] LOG: database system is shut down 2025-03-27 19:55:32.860 UTC [1] FATAL: could not load private key file "/var/lib/postgresql/certs/server.key": Permission denied 2025-03-27 19:55:32.860 UTC [1] LOG: database system is shut down

This is in spite that the persistent storage states that the destination path is

Originally posted by @Tmmcmasters in https://github.com/coollabsio/coolify/discussions/3976#discussioncomment-12645665

[andrasbacsai]

The actual files are located on the server /data/coolify/database...?

[Tmmcmasters]

The actual files are located on the server /data/coolify/database...?

Will run some testing on this and let you know

[Tmmcmasters]

The actual files are located on the server /data/coolify/database...?

@andrasbacsai I ran some testing and I am unable to actually change the file mount source path and destination path. I went to the root server and I am running this on localhost. It appears that it is there where it says in the source path.

I am unable to see the destination path because the Postgres server keeps shutting down and the terminal is inaccessible.

[peaklabs-dev]

Can you use SSH and then check the ssl folder and do an ls in the ssl folder please.

[Tmmcmasters]

Can you use SSH and then check the ssl folder and do an ls in the ssl folder please.

Will take a look at this later this evening. Thank you

[FlashSkyNews]

Hey! Bumping this issue because I'm also experiencing this.

I've noticed, that when creating the postgres instance it tries to assign something to the postgres user I've defined. This user doesn't exist tho. Deploying the instance with default values (user=postgres) works tho.

[Tmmcmasters]

I think the issue is related to the user as well, the folder structure appears to be fine....

[ApoloApps]

same problem here, using custom user (maybe created custom user does not have admin access??? idk)

Logs

2025-04-08 13:55:57.961 UTC [1] FATAL: could not load private key file "/var/lib/postgresql/certs/server.key": Permission denied 2025-04-08 13:55:57.961 UTC [1] LOG: database system is shut down

[W2Wizard]

Bump, same here. If you simply keep the user as is then it works, otherwise it breaks.

[codelonesomest]

I also got this problem after regenerate ssl certificates. I assume the command cause the problem?

volumes:
            - 'postgres-data-co000kw0c8ooos8oowcg4g4g:/var/lib/postgresql/data'
            - '/data/coolify/databases/co000kw0c8ooos8oowcg4g4g/ssl/server.pem:/var/lib/postgresql/certs/server.pem'
        command:
            - postgres
            - '-c'
            - ssl=on
            - '-c'
            - ssl_cert_file=/var/lib/postgresql/certs/server.crt
            - '-c'
            - ssl_key_file=/var/lib/postgresql/certs/server.key

[scherddel]

I had the same issue but only after I have created the Postgres Instance. (Where i added the "Enable SSL" after the creation as a second step) Can you try to create a new one and select "Enable SSL" from the start?

This resolved it for me. Still I assume there is an issue in the modification process.

[Megajin]

@andrasbacsai @peaklabs-dev

I've noticed that files created manually under /data/coolify/databases/<uid>/docker-entrypoint-initdb.d as well as changes made to the generated docker-compose.yml are being overwritten. This makes it difficult to apply custom fixes or adjustments in environments where permission or ownership issues occur.

I’ve also attempted to use the initialization script functionality on the PostgreSQL database page, but it seems that only plain text is allowed — scripts including #!/bin/bash are not used. Would it be possible to change this behavior via a toggle or configuration option?

It would be extremely helpful to allow simple initialization scripts to be injected during container startup. For example, a script like the following could quickly solve recurring permission issues:

#!/bin/bash

chown postgres:postgres /var/lib/postgresql/certs/server.crt
chown postgres:postgres /var/lib/postgresql/certs/server.key
chown postgres:postgres -R /var/lib/postgresql/data

Additionally, it seems there's a broader issue within the PHP-based actions: when changing the default database user from postgres to a different one, the system breaks due to unexpected permission problems.

Supporting script-based customization and avoiding automatic overwrites would greatly increase flexibility for teams managing Coolify in more complex setups.

Would love to hear your thoughts on this!

[scherddel]

@Megajin While adding customization options can certainly help work around issues, the core problem appears to be with permission handling during modifications - an issue that should be addressed first as it's causing cascading problems.

This seems a related bug #4519 that was previously reported: https://github.com/coollabsio/coolify/issues/4519

Most users rely on Coolify being robust out-of-the-box and likely prefer not to troubleshoot or customize configurations just to achieve expected functionality. Addressing the underlying permission handling would provide a more sustainable solution.

[Megajin]

@scherddel I agree. However as long as there is "beta" in the release name, I would appreciate an option to tinker and quickfix such problems myself. But your point makes absolutely sense and that should be the way.

[pawelkrystkiewicz]

I'm also facing the same issue. When can we expect a fix?

[GauthierPLM]

I'm also experiencing the same issue. The certificates should be assigned to either root or the postgres user (uid 70 gid 70), but instead the files are owned by the user used by Coolify to connect to my server, as I use Coolify with a non-root user. This prevent postgres from reading the files.

If I manually change the rights in /data/coolify/databases/<id>/ssl, it will be erased the next time the database is restarted.

[eersnington]

I'm facing this issue too in v4.0.0-beta.418. Recreated it in the video below

I launched a postgres url, then made it publicly available in 6543 (5432 is being used) and then stopped it to enable SSL. This causes a proxy sheet component to open and I wont be able to close it unless I refresh the app.

Then after starting it, the resource becomes degraded.

Logs

2025-05-29 20:18:22.718 UTC [1] FATAL:  could not load private key file "/var/lib/postgresql/certs/server.key": Permission denied
2025-05-29 20:18:22.718 UTC [1] LOG:  database system is shut down

https://github.com/user-attachments/assets/3aef9646-bdba-4bd9-b3ae-c1341cc1ff05

Weirdly, this issue isn't there when I create a new postgres resource and enable ssl from the get go before launching it. My guess it there's some permission issue for the certs path? or that it's being created in the wrong dir?

[oliverfrost1]

Also getting this issue: 20:09:26.405 UTC [1] FATAL: could not load private key file "/var/lib/postgresql/certs/server.key": Permission denied

[Fayston]

Until a fix, here is a workaround:

1. Create a directory for the certificates

   mkdir /data/coolify/databases/DATABASE_ID/ssl
   

2. Handle your SSL certificate and key ensuring they are named server.key and server.crt:

   • If you don't have a certificate and key: Copy the ones generated by Coolify in the UI into the /data/coolify/databases/DATABASE_ID/ssl/ directory
   • If you already have your own: Copy your files into /data/coolify/databases/DATABASE_ID/ssl/

3. Set ownership and permissions:

   chmod 600 /data/coolify/databases/DATABASE_ID/ssl/*
   chown 999:999 /data/coolify/databases/DATABASE_ID/ssl/*
   

4. Go to your postgresql resource "Persistent Storage" section in the UI

5. Delete auto-generated file mounts from "Persistent Storage" that point to the key and crt files

6. Add a new "Directory Mount" :

   • Source Directory: /data/coolify/databases/DATABASE_ID/ssl
   • Destination Directory: /var/lib/postgresql/certs

[bondbenz]

Same issue @Fayston

[douglasg14b]

Also running into this with a new postgres DB

This started happening after removing a DB, and then creating another one in the same project with the same name.

Oof, dos this mean that DBs effectively cannot/do not work OOTB with SSL? That's a big deal IMHO

[douglasg14b]

@Fayston Unfortunately the guide does not appear to work. Still get Permission denied errors :/

During DB startup I do see a chown failure

 Container tkoccg0c04www8wcc4o048kw  Created
 Container tkoccg0c04www8wcc4o048kw  Starting
 Container tkoccg0c04www8wcc4o048kw  Started
chown: unknown user/group discord_bot_user:discord_bot_user

Indicating it's trying to change ownership to a user/group that doesn't exist.

[prenansantana]

Add a new "Directory Mount" :

• Source Directory: /data/coolify/databases/DATABASE_ID/ssl
• Destination Directory: /var/lib/postgresql/certs

It works for me, thanks. It is necessary add before start database first time

[scherddel]

Add a new "Directory Mount" :

• Source Directory: /data/coolify/databases/DATABASE_ID/ssl
• Destination Directory: /var/lib/postgresql/certs

It works for me, thanks. It is necessary add before start database first time

Now edit it :-) -> I think this is the issue we are talking about not the initial setup

[viraj-lead]

Anyone has a workaround that works? Re-created the database as @Fayston mentioned but it replaces the existing certs that I manually add.