charts [bitnami/postgresql] Better documentation needed on major version upgrades

Name and Version

bitnami/postgresql 12.2.0

What is the problem this feature will solve?

Currently the upgrading section of bitnami/postgresql is simply an external link to the PostgreSQL docs on how to upgrade major versions. This is insufficient for providing a bare minimum level of information on how to upgrade the PostgreSQL major version when using the Bitnami PostgreSQL Helm chart.

I am opening this issue to suggest that Bitnami could be more prescriptive and detailed as to how to upgrade the postgresql chart + image major version, giving specific consideration to PostgreSQL being deployed to Kubernetes, beyond simply linking to the PostgreSQL docs.

A well known paint point of PostgreSQL is that major-version upgrades require generally one of two steps to un-break things: pg_dumpall or pg_upgrade. In the absence of these, the PostgreSQL upgrade will fail with FATAL database files are incompatible with server and indicate that

The data directory was initialized by PostgreSQL version <old-version>,
which is not compatible with this version <old-version>.

The Bitnami documentation could at the very least make a recommendation between one of these two methods, or provide a complete, working, and minimal example for upgrading.

What is the feature you are proposing to solve the problem?

To more easily upgrade the PostgreSQL major version, either because we have bumped the Bitnami chart major version or explicitly bumped the postgresql.image.tag.

What alternatives have you considered?

https://github.com/tianon/docker-postgres-upgrade
https://github.com/bitnami/charts/issues/8025#issuecomment-964906018

Feb 16 '23 18:02 brsolomon-deloitte

Here is a full test of https://github.com/bitnami/charts/issues/8025#issuecomment-964906018 that ends in failure:

kubectl create namespace pgtest
argocd app create pgtest \
    --core \
    --dest-namespace pgtest \
    --repo https://charts.bitnami.com/bitnami \
    --helm-chart postgresql \
    --revision '11.9.13' \
    --dest-server https://kubernetes.default.svc \
    --helm-set-string 'auth.postgresPassword=098a981677'
argocd app sync --assumeYes --core pgtest

kubectl logs --tail 10 -n pgtest sts/pgtest-postgresql
# postgresql 18:46:45.53 INFO  ==> ** PostgreSQL setup finished! **

# postgresql 18:46:45.54 INFO  ==> ** Starting PostgreSQL **
# 2023-02-16 18:46:45.559 GMT [1] LOG:  pgaudit extension initialized
# 2023-02-16 18:46:45.566 GMT [1] LOG:  starting PostgreSQL 14.5 on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
# 2023-02-16 18:46:45.566 GMT [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
# 2023-02-16 18:46:45.566 GMT [1] LOG:  listening on IPv6 address "::", port 5432
# 2023-02-16 18:46:45.570 GMT [1] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
# 2023-02-16 18:46:45.574 GMT [131] LOG:  database system was shut down at 2023-02-16 18:46:45 GMT
# 2023-02-16 18:46:45.580 GMT [1] LOG:  database system is ready to accept connections

kubectl exec -it -n pgtest sts/pgtest-postgresql -- bash
I have no name!@pgtest-postgresql-0:/$ id
uid=1001 gid=0(root) groups=0(root),1001
I have no name!@pgtest-postgresql-0:/$ postgres --version
postgres (PostgreSQL) 14.5
I have no name!@pgtest-postgresql-0:/$ printenv POSTGRES_PASSWORD
098a981677
I have no name!@pgtest-postgresql-0:/$ PGPASSWORD=$POSTGRES_PASSWORD psql --host 127.0.0.1 -U postgres -d postgres -p 5432
psql (14.5)
Type "help" for help.

postgres=# CREATE TABLE IF NOT EXISTS app_user (
postgres(#   username varchar(45) NOT NULL,
postgres(#   password varchar(450) NOT NULL,
postgres(#   enabled integer NOT NULL DEFAULT '1',
postgres(#   PRIMARY KEY (username)
postgres(# );
CREATE TABLE
postgres=# exit


argocd app set pgtest --core --revision '12.2.0'

argocd app set pgtest --core \
    --helm-set 'primary.containerSecurityContext.runAsUser=0' \
    --helm-set 'diagnosticMode.enabled=true' \
    --helm-set-string 'auth.postgresPassword=098a981677'

$ argocd app diff --core pgtest

===== /Secret pgtest/pgtest-postgresql ======
12c12
<     helm.sh/chart: postgresql-11.9.13
---
>     helm.sh/chart: postgresql-12.2.0

===== /Service pgtest/pgtest-postgresql ======
12c12
<     helm.sh/chart: postgresql-11.9.13
---
>     helm.sh/chart: postgresql-12.2.0

===== /Service pgtest/pgtest-postgresql-hl ======
12c12
<     helm.sh/chart: postgresql-11.9.13
---
>     helm.sh/chart: postgresql-12.2.0

===== apps/StatefulSet pgtest/pgtest-postgresql ======
13c13
<     helm.sh/chart: postgresql-11.9.13
---
>     helm.sh/chart: postgresql-12.2.0
226c226
<         helm.sh/chart: postgresql-11.9.13
---
>         helm.sh/chart: postgresql-12.2.0
238,239d237
<               namespaces:
<               - pgtest
243c241,245
<       - env:
---
>       - args:
>         - infinity
>         command:
>         - sleep
>         env:
245c247
<           value: "false"
---
>           value: "true"
273c275
<         image: docker.io/bitnami/postgresql:14.5.0-debian-11-r35
---
>         image: docker.io/bitnami/postgresql:15.2.0-debian-11-r0
275,285d276
<         livenessProbe:
<           exec:
<             command:
<             - /bin/sh
<             - -c
<             - exec pg_isready -U "postgres" -h 127.0.0.1 -p 5432
<           failureThreshold: 6
<           initialDelaySeconds: 30
<           periodSeconds: 10
<           successThreshold: 1
<           timeoutSeconds: 5
291,304d281
<         readinessProbe:
<           exec:
<             command:
<             - /bin/sh
<             - -c
<             - -e
<             - |
<               exec pg_isready -U "postgres" -h 127.0.0.1 -p 5432
<               [ -f /opt/bitnami/postgresql/tmp/.initialized ] || [ -f /bitnami/postgresql/.initialized ]
<           failureThreshold: 6
<           initialDelaySeconds: 5
<           periodSeconds: 10
<           successThreshold: 1
<           timeoutSeconds: 5

argocd app sync --core pgtest


# now do the upgrade

kubectl exec -it -n pgtest sts/pgtest-postgresql -- bash

postgres --version
. /opt/bitnami/scripts/libos.sh
ensure_group_exists postgres -i 1002
ensure_user_exists postgres -i 1002 -g postgres
mv /bitnami/postgresql/data /bitnami/postgresql/olddata
mkdir -p /bitnami/postgresql/data /bitnami/postgresql/oldbin
chown -R postgres:postgres /bitnami/postgresql/data /bitnami/postgresql/olddata /bitnami/postgresql/oldbin
cd /tmp
# get the old version
apt-get update -y && apt-get install -y --no-install-recommends curl
export OLD_PG_VERSION='14.7.0-0'
export OLD_PG_VERSION_MMP='14.7.0'
curl --remote-name --silent https://downloads.bitnami.com/files/stacksmith/postgresql-${OLD_PG_VERSION}-linux-amd64-debian-11.tar.gz
tar --extract --directory /bitnami/postgresql/oldbin/ \
    --file postgresql-${OLD_PG_VERSION}-linux-amd64-debian-11.tar.gz \
    --strip-components=4 \
    postgresql-${OLD_PG_VERSION_MMP}-linux-amd64-debian-11/files/postgresql/bin
ls /bitnami/postgresql/oldbin/
gosu postgres initdb -E UTF8 -D /bitnami/postgresql/data -U postgres
gosu postgres pg_upgrade -c -b /bitnami/postgresql/oldbin -B /opt/bitnami/postgresql/bin -d /bitnami/postgresql/olddata -D /bitnami/postgresql/data

Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok

*failure*
Consult the last few lines of "/bitnami/postgresql/data/pg_upgrade_output.d/20230216T192921.303/log/pg_upgrade_server.log" for
the probable cause of the failure.

connection to server on socket "/tmp/.s.PGSQL.50432" failed: No such file or directory
    Is the server running locally and accepting connections on that socket?

could not connect to source postmaster started with the command:
"/bitnami/postgresql/oldbin/pg_ctl" -w -l "/bitnami/postgresql/data/pg_upgrade_output.d/20230216T192921.303/log/pg_upgrade_server.log" -D "/bitnami/postgresql/olddata" -o "-p 50432 -b  -c listen_addresses='' -c unix_socket_permissions=0700 -c unix_socket_directories='/tmp'" start
Failure, exiting

root@pgtest-postgresql-0:/tmp# cat /bitnami/postgresql/data/pg_upgrade_output.d/20230216T192921.303/log/pg_upgrade_server.log
-----------------------------------------------------------------
  pg_upgrade run on Thu Feb 16 19:29:21 2023
-----------------------------------------------------------------

command: "/bitnami/postgresql/oldbin/pg_ctl" -w -l "/bitnami/postgresql/data/pg_upgrade_output.d/20230216T192921.303/log/pg_upgrade_server.log" -D "/bitnami/postgresql/olddata" -o "-p 50432 -b  -c listen_addresses='' -c unix_socket_permissions=0700 -c unix_socket_directories='/tmp'" start >> "/bitnami/postgresql/data/pg_upgrade_output.d/20230216T192921.303/log/pg_upgrade_server.log" 2>&1
waiting for server to start....postgres: could not access the server configuration file "/bitnami/postgresql/olddata/postgresql.conf": No such file or directory
 stopped waiting
pg_ctl: could not start server
Examine the log output.

Feb 16 '23 19:02 brsolomon-deloitte

Hi @brsolomon-deloitte,

We are open to contributions! If you want you can create a PR with the things you think are necessary to upgrade, we will be happy to review it!

Feb 20 '23 09:02 Mauraza

Hi @brsolomon-deloitte,

We are open to contributions! If you want you can create a PR with the things you think are necessary to upgrade, we will be happy to review it!

I don't have much to contribute here since my attempt above ends in an error. I would think that the Bitnami team would have a full reproducible solution here with the latest postgres chart and images (v 14.x/15.x). That's why I opened this issue.

Feb 20 '23 14:02 brsolomon-deloitte

I execute the following command and there is an error in my minikube：

helm repo add bitnami https://charts.bitnami.com/bitnami
helm install my-release bitnami/keycloak

The data directory was initialized by PostgreSQL version 11, which is not compatible with this version 15.2

Feb 21 '23 07:02 hanbinloop

@Mauraza

Feb 21 '23 07:02 hanbinloop

Hi @hanbinloop,

Could you create another issue with this? This issue is related to the PostgreSQL Chart no with Keycloak. Thanks!

Feb 21 '23 08:02 Mauraza

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

Mar 10 '23 01:03 github-actions[bot]

not stale

Mar 10 '23 11:03 brsolomon-deloitte

Hi @brsolomon-deloitte,

I have created a task internally to review this error. We currently have a lot on our plate. For this reason, we will update this thread when we update the documentation.

Mar 10 '23 16:03 Mauraza

bump (anti-stale message)

Mar 31 '23 14:03 hatharom

This seems a serious issue.

Because of lack of proper documentation and/or built-in solution I was forced to do some hand-made hack like: modifiying the image version then stepping into the pod and perform pg_upgrade or pg_dumpall according to this https://github.com/bitnami/charts/issues/8025#issuecomment-964906018

Needless to say non of them worked out pg_upgrade fails because of OS version change between the postgresimage v11 and v15 pg_dump fails because of permission stuff... I got tired to debug further

Native postgres upgrade seems a piece of cake compared to this bitnamichart ecosystem....

Mar 31 '23 16:03 hatharom

Hi @hatharom,

As appears in the before comment , We created a task to improve the documentation. We can't provide an ETA. It depends on the team's capacity, and other initiatives we are working on. We will update this thread once done.

Apr 03 '23 08:04 Mauraza

its really a shame that there is still no migration path introducing a lot of breaking changes and not providing any docs how to upgrade is not really professional

i'm really disappointed here

May 25 '23 15:05 eloo-abi

Hi @eloo-abi,

Sorry to read this. Always consult the official documentation. As I said before we have a task to improve the documentation but we don't know when we will be able to work on this because it depends on the capacity of the team.

If anyone wants to contribute to this, you are more than welcome. We will be happy to review it.

May 29 '23 08:05 Mauraza

I execute the following command and there is an error in my minikube：
helm repo add bitnami https://charts.bitnami.com/bitnami
helm install my-release bitnami/keycloak
The data directory was initialized by PostgreSQL version 11, which is not compatible with this version 15.2

Hey @hanbinloop did you ever figure this out or recover? I got rugpulled by the bitnami/postgres container in bitnami/keycloak too I was just performing a routine upgrade to some ingress annotations and it upgraded the whole postgres container on me. Trying to figure out how to get unstuck.

Oct 11 '23 00:10 Mbd06b

I execute the following command and there is an error in my minikube：
helm repo add bitnami https://charts.bitnami.com/bitnami
helm install my-release bitnami/keycloak
The data directory was initialized by PostgreSQL version 11, which is not compatible with this version 15.2
Hey @hanbinloop did you ever figure this out or recover? I got rugpulled by the bitnami/postgres container in bitnami/keycloak too I was just performing a routine upgrade to some ingress annotations and it upgraded the whole postgres container on me. Trying to figure out how to get unstuck.

Keycloak only supports Postgres 15 anyways https://www.keycloak.org/server/db so just set the postgres image to 15 or manage via external database.

Oct 12 '23 13:10 mfisch04

The data directory was initialized by PostgreSQL version <old-version>,
which is not compatible with this version <old-version>.

This problem affects all charts that use postgresql and the manual steps are difficult to get right. I still search for an easy fix

https://www.postgresql.org/docs/16/upgrading.html https://github.com/bitnami/charts/issues/8025#issuecomment-1708879837 https://stackoverflow.com/questions/64850625/how-to-upgrade-postgresql-inside-a-kubernetes-pod https://stackoverflow.com/questions/69898410/upgrade-bitnami-postgresql-image-k8s/69899094#69899094 https://goauthentik.io/docs/troubleshooting/postgres/upgrade_kubernetes

Oct 16 '23 11:10 ghost

what about keycloak just set:

postgresql:
  image:
    tag: 15.4.0-debian-11-r54

I don't think it's a postgres chart problem.

Oct 22 '23 21:10 batazor

what about Keycloak just set:

Sure, this is indeed something that Bitnami recommends--to explicitly set the version of dependencies in dependency charts.

I don't think it's a postgres chart problem.

Yes, it is. Pinning the chart version doesn't solve the problem of the eventual upgrade that would be needed. Would you keep the pinned postgresql chart to an old major version that is no longer receiving security updates? No, probably not. The problem remains that better documentation and automation is needed around how to upgrade PostgreSQL major version within the chart.

Oct 23 '23 10:10 brsolomon-deloitte

Hi all, Same issue here, any solution/documentation to help on that ? The last 17.x.x release for keycloak use the new version of postgre, but no documentation is provided to migrate pg 15 to 16.

Nov 08 '23 13:11 Gwojda

what about keycloak just set:
postgresql:
  image:
    tag: 15.4.0-debian-11-r54
I don't think it's a postgres chart problem.

This helped as a stopgap. Might also be worth noting this comment, though that wasn't necessarily applicable, since I have a source of truth for my secrets.

I haven't actually started on a migration path, but after a quick look at the official keycloak docs, I'd imagine it'd go something like:

backup your data
create external uplevel postgres deployment
helm upgrade but point host to external uplevel postgres deployment
let automatic migration run
helm upgrade again without the external DB configs
stream pg dump from external uplevel postgres deployment into default uplevel postgres deployment
soak time
done!

Certainly doable, but would be nice if the keycloak folks had something like this out of the box.

Nov 10 '23 21:11 cjvirtucio87

I successfully updated from Chart v15 to v17 (and so from postgresql 15 to 16) using the following process:

Update Chart version, keeping the same postgresql version (for now) by editing the values.yaml.
Add a new fresh and separated postgresql instance with latest version, using the same credentials

Chart.yaml

apiVersion: v2
name: keycloak
type: application
version: 1.0.0
appVersion: "1.0.0"
dependencies:
- name: keycloak
  version: 17.3.1
  repository: https://charts.bitnami.com/bitnami
- name: postgresql
  alias: postgresql-16
  version: 13.2.9
  repository: https://charts.bitnami.com/bitnami

values.yaml

keycloak:
[...]
  postgresql:
    enabled: true
    image:
      registry: docker.io
      repository: bitnami/postgresql
      tag: 15.3.0-debian-11-r74
[...]
postgresql-16:
  architecture: standalone
  auth:
    username: keycloak
    database: keycloak
    existingSecret: keycloak-psql-secret

connect on current postgresql pod and backup:

export PGPASSWORD=$POSTGRES_PASSWORD
pg_dumpall -U postgres > bitnami/postgresql/backup.sql

copy the file into your new postgresql:

kubectl cp -n keycloak keycloak-postgresql-0:/bitnami/postgresql/backup.sql /tmp/backup.sql
kubectl cp -n keycloak /tmp/backup.sql keycloak-postgresql-16-0:/bitnami/postgresql/backup.sql

connect to new postgresql pod and restore:

export PGPASSWORD=$POSTGRES_PASSWORD
psql -U postgres -h localhost -f /bitnami/postgresql/backup.sql

be sure to check you have a key named db-password which match the password key in your psql secret. For any reason, the chart is not using the same key for internal vs external db:

$ kubectl get secret -n keycloak keycloak-psql-secret -o yaml
apiVersion: v1
data:
  db-password: ****
  password: ****
  postgres-password: ****
kind: Secret
[...]

Change the configuration of your keycloak:

[...]
  postgresql:
    enabled: false ############################ Disable the current DB
    image:
      registry: docker.io
      repository: bitnami/postgresql
      tag: 15.3.0-debian-11-r74
    architecture: standalone
    auth:
      username: keycloak
      database: keycloak
      existingSecret: keycloak-psql-secret
  externalDatabase: ########################### Setup the new one
    host: keycloak-postgresql-16
    user: keycloak
    database: keycloak
    existingSecret: keycloak-psql-secret

I know It's not ideal, but it's working and is the only way I could safely update it. Also it should not lead to any downtime, except during restart if you're not in HA. You will just have to ensure that no data are created between backup and restore procedures.

Nov 17 '23 08:11 headyj

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

Dec 06 '23 01:12 github-actions[bot]

Not stale

Dec 06 '23 01:12 brsolomon-deloitte

Do we already have an idea how to best solve this?

My instinct would be to add an init container that checks the current version against the version of the datafiles on disk.

If they match 
-> Launch the normal postgres container as always

If they do not match
  when version_on_disk is exactly one (1) below the container version
  	create a backup
  	perform the upgrade
  	  when this fails
  	  	restore the backup to the old location
  	    abort
  	  on success		
  		-> Launch the normal postgres container as always
  when not
  	abort

I'd have made this behaviour optional.

Dec 06 '23 16:12 4censord

Am Wed, Dec 06, 2023 at 08:25:19AM -0800 schrieb 4censord:

Do we already have an idea how to best solve this?

My instinct would be to add an init container that checks the current version against the version of the datafiles on disk.
If they match
-> Launch the normal postgres container as always

If they do not match
 when version_on_disk is exactly one (1) below the container version
 	create a backup
 	perform the upgrade
 	  when this fails
 	  	restore the backup to the old location
 	    abort
 	  on success		
 		-> Launch the normal postgres container as always
 when not
 	abort
I'd have added this as an optional behaviour to the chart. This sounds great! It might require a bit additional space. Should that be a separate volume?

Dec 06 '23 23:12 christf

I ran into this using the bitnami/mastodon chart. I landed up disabling the Postgresql installation bundled with mastodon and using a separate Postgresql. Now I have 2 helm charts to update but I suppose now that I've migrated the data to new new version I I could migrate my data back and reenable using the Mastodon provided one.

My older Postgresql 15 pod was named mastodon-postgresql-0 and the fresh installed Postresql 16 had a service named postgresql. I was able to exec into mastodon-postgresql-0, dump the database, and import into the new install by running the following:

exec --stdin --tty mastodon-postgresql-0 -- /opt/bitnami/scripts/postgresql/entrypoint.sh /bin/bash

#Run the following command to dump the postgresql DB
PGPASSWORD='MY_SECRET_PASSWORD' pg_dump --create -U postgres bitnami_mastodon > /bitnami/postgresql/mastodon.sql 

#Upgrade the data to use Postgres 16
PGPASSWORD="MY_OTHER_SECRET_PASSWORD" psql -U postgres --host postgresql < /bitnami/postgresql/mastodon.sql

I then updated the Mastodon chart to disable the provided Postgresql and use my external installation for version 16.

I didn't know how to make helm upgrade do this. It's essentially what @headyj did in https://github.com/bitnami/charts/issues/14926#issuecomment-1815926198 above.

Dec 06 '23 23:12 joelparker

I wrote a script for automatic major version upgrade bitnami/postgres-ha based on job under k8s for reference.

https://github.com/exfly/bitnami-pg-upgrade https://exfly.github.io/postgres-upgrade-bitnami-ha/

Dec 25 '23 04:12 exfly

For more complex scenarios (such as in-place major version upgrades), perhaps it would be good to consider an operator rather than this helm chart?

Examples: https://github.com/cloudnative-pg/cloudnative-pg https://github.com/zalando/postgres-operator https://github.com/CrunchyData/postgres-operator

Jan 08 '24 13:01 JosefWN

I successfully updated from Chart v15 to v17 (and so from postgresql 15 to 16) using the following process:

* Update Chart version, keeping the same postgresql version (for now) by editing the values.yaml.

* Add a new fresh and separated postgresql instance with latest version, using the same credentials

Chart.yaml

apiVersion: v2
name: keycloak
type: application
version: 1.0.0
appVersion: "1.0.0"
dependencies:
- name: keycloak
  version: 17.3.1
  repository: https://charts.bitnami.com/bitnami
- name: postgresql
  alias: postgresql-16
  version: 13.2.9
  repository: https://charts.bitnami.com/bitnami

values.yaml

keycloak:
[...]
  postgresql:
    enabled: true
    image:
      registry: docker.io
      repository: bitnami/postgresql
      tag: 15.3.0-debian-11-r74
[...]
postgresql-16:
  architecture: standalone
  auth:
    username: keycloak
    database: keycloak
    existingSecret: keycloak-psql-secret

* connect on current postgresql pod and backup:

export PGPASSWORD=$POSTGRES_PASSWORD
pg_dumpall -U postgres > bitnami/postgresql/backup.sql

* copy the file into your new postgresql:

kubectl cp -n keycloak keycloak-postgresql-0:/bitnami/postgresql/backup.sql /tmp/backup.sql
kubectl cp -n keycloak /tmp/backup.sql keycloak-postgresql-16-0:/bitnami/postgresql/backup.sql

* connect to new postgresql pod and restore:

export PGPASSWORD=$POSTGRES_PASSWORD
psql -U postgres -h localhost -f /bitnami/postgresql/backup.sql

* be sure to check you have a key named `db-password` which match the `password` key in your psql secret. For any reason, the chart is not using the same key for internal vs external db:

$ kubectl get secret -n keycloak keycloak-psql-secret -o yaml
apiVersion: v1
data:
  db-password: ****
  password: ****
  postgres-password: ****
kind: Secret
[...]

* Change the configuration of your keycloak:

[...]
  postgresql:
    enabled: false ############################ Disable the current DB
    image:
      registry: docker.io
      repository: bitnami/postgresql
      tag: 15.3.0-debian-11-r74
    architecture: standalone
    auth:
      username: keycloak
      database: keycloak
      existingSecret: keycloak-psql-secret
  externalDatabase: ########################### Setup the new one
    host: keycloak-postgresql-16
    user: keycloak
    database: keycloak
    existingSecret: keycloak-psql-secret

I know It's not ideal, but it's working and is the only way I could safely update it. Also it should not lead to any downtime, except during restart if you're not in HA. You will just have to ensure that no data are created between backup and restore procedures.

this is what I did, as well. it's looking like relying on the embedded postgresql for keycloak isn't viable long-term. I also opted for neither using this chart nor the embedded postgresql on the keycloak chart and just managing postgresql entirely by myself.

slightly different steps that I took that's potentially a bit more scriptable:

Dump the database:

kubectl --namespace sso-dev exec --stdin keycloakx-postgresql-0 -- sh -c "PGPASSWORD=${KEYCLOAKX_POSTGRESQL_POSTGRES_PASSWORD} pg_dumpall --username=postgres --host=127.0.0.1 --port=5432 | base64" > /tmp/keycloakx-postgresql.sql.base64

Scale down the statefulsets/keycloakx-postgresql:

kubectl --namespace sso-dev scale --replicas=0 statefulsets/keycloakx-postgresql

Delete the pvc:

kubectl --namespace sso-dev delete pvc/data-keycloakx-postgresql-0

Bump up the postgresql image version and redeploy

Restore the database:

kubectl --namespace sso-dev exec --stdin keycloakx-postgresql-0 -- sh -c "base64 -d | PGPASSWORD=${KEYCLOAKX_POSTGRESQL_POSTGRES_PASSWORD} psql --username=postgres --host=127.0.0.1 --port=5432 -f -" < /tmp/keycloakx-postgresql.sql.base64

Feb 11 '24 14:02 cjvirtucio87