backup-container icon indicating copy to clipboard operation
backup-container copied to clipboard

Add support for the Patroni Spilo database for backups

Open ikethecoder opened this issue 4 years ago • 1 comments

ikethecoder avatar Dec 14 '20 22:12 ikethecoder

Hi @WadeBarnes I was reluctant to create a separate image specifically for the backup-container - those scripts under "/patroni-spilo/root/" are not part of the Spilo base image, they are only needed to get the backup.sh to work. I'll trim the scripts to bare minimum to see what is left - if its still too much then I suppose can pull it out into a bcgov repo.

I had quite a bit of trouble getting backup-container to work with a database dump from a Spilo-based Patroni cluster using the existing Postgres image (I never got it working cleanly for full cycle of backup/validate/restore). Spilo adds some specific extensions (pg_stat_statements,pg_stat_kcache,set_user) that were not in the Postgres 12 base image used in backup-container, which caused the validate database phase to fail - partly for missing extensions and partly for missing Roles (admin, robot_zmon) on Grants). This is what led me down the path of starting with Spilo in the backup image, and then adding in the "/root/*" bundle of scripts like "run-postgresql" to get backup.sh to work.

I have been looking at both the "platform services" Patroni image/deployment and the Spilo Patroni image/deployment - and I have found Spilo to be very stable with it handling graceful shutdown nicely when Pods are terminated - something that the platform patroni image doesn't seem to do well.

ikethecoder avatar Dec 15 '20 18:12 ikethecoder