The data directory was initialized by PostgreSQL version 16, which is not compatible with this version 17.0

[f3k-freek]

Describe the issue

I get the following fatal error in the logs when upgrading from helm chart 0.5.16 to 0.5.17:

postgresql 12:51:27.33 INFO  ==> ** PostgreSQL setup finished! **
postgresql 12:51:27.43 INFO  ==> ** Starting PostgreSQL **
2024-11-04 12:51:27.538 GMT [1] FATAL:  database files are incompatible with server
2024-11-04 12:51:27.538 GMT [1] DETAIL:  The data directory was initialized by PostgreSQL version 16, which is not compatible with this version 17.0.

I already tried

• [X] I've read and searched the documentation.
• [X] I've searched for similar filed issues in this repository.

Steps to reproduce the behavior

• Install helm chart 0.5.16 with persistency.
• Upgrade to helm chart 0.5.17

Expected behavior

It should upgrade the helm chart without issues or human intervention.

Screenshots

No response

Exception traceback

No response

Additional context

No response

[f3k-freek]

I've solved the issue for now by adding the following to the value file:

postgresql:
  image:
    tag: "16.4.0-debian-12-r9"

It would be really nice if there would be warnings and upgrade instructions as long as the helm chart doesn't handle the upgrades automatically

[github-actions[bot]]

This issue has been automatically marked as stale because there wasn’t any recent activity.

It will be closed soon if no further action occurs.

Thank you for your contributions!

[f3k-freek]

The upgrade process for the Bitnami PostgreSQL chart can be challenging and lacks sufficient documentation. Below is the procedure I followed for a successful upgrade:

Before Upgrading:

1. Prevent Frontend Changes:

   • Set replicaCount: 0 in the values.yml file to scale down the frontend and prevent any changes during the upgrade process.

2. Backup the Database:

   • Enter the shell of the existing PostgreSQL pod:

     kubectl exec -it <postgresql-pod-name> -- bash
     

   • Create a PostgreSQL dump of the database and save it to the persistent directory:

     pg_dumpall -U postgres > /bitnami/postgresql/backup.sql
     

3. Prepare for Upgrade:

   • Rename the current PostgreSQL data directory to ensure it is preserved in case of issues after the upgrade:

     mv /bitnami/postgresql/data /bitnami/postgresql/data-old
     

Upgrading the Helm Chart:

4. Upgrade the Helm Chart:

   • Upgrade the Weblate chart to the desired version.

5. Restore the Database:

   • After upgrading the Helm chart, enter the shell of the newly upgraded PostgreSQL pod:

     kubectl exec -it <new-postgresql-pod-name> -- bash
     

   • Restore the backup by running the following command:

     psql -U postgres -h localhost -f /bitnami/postgresql/backup.sql
     

6. Restore Frontend Availability:

   • Set replicaCount: 1 in the values.yml file to scale the frontend back up and make it available again.

Post-Upgrade Cleanup:

7. Verify the Upgrade:

   • Ensure everything is working as expected by checking that the PostgreSQL instance is running correctly and the frontend is accessible.

8. Remove Backup Files:

   • Once verified, enter the shell of the PostgreSQL pod again and remove the backup files to clean up:

     rm /bitnami/postgresql/backup.sql
     rm -r /bitnami/postgresql/data-old
     

[eestolano]

I came across a third-party postgres Docker container that's supposed to handle upgrades. It wasn't originally built to work with the Bitnami containers, but there's currently an open PR for Bitnami support that's awaiting testing: https://github.com/pgautoupgrade/docker-pgautoupgrade/pull/66 I have not yet tried it myself.

[justinclift]

Yeah, it turns out Bitnami does some interesting stuff which means our (pgautoupgrade devs) standard images (as they currently are) don't work with it out of the box.

@andyundso is working through the Bitnami specific issues to get it working, and if someone has time to test the in progress image (NOT on your production databases!) that would be super helpful.

Anyone up for it? :smile: