Restore database from pv backup

[cslovell]

Hi - first of all, fantastic repository. Great to see all the hard work you've put into this.

I have in mind a scenario and would like to know to what extent this is supported already or would need to be a feature request:

After a catastrophic event, the entire cluster is deleted. However, the administrator specified a reclaim policy of retain on the storage class that backed the original database volumes, and the cloud provider has retained these volumes, which are immediately re-mounted to a new cluster as persistent volumes. Since the old cluster has been destroyed, the administrator would like to quickly restore the database by referencing the new volume he just created in a new Kubegres deployment.

Something like this comes to mind:

apiVersion: kubegres.reactive-tech.io/v1
kind: Kubegres
...
restoreFromVolume: [pv-name]

Thanks!

[alex-arica]

Hi, thank you for your kind feedback.

Currently the way we do restoration is manual: we create a new cluster of Postrgres with Kubegres with one Primary only (no replicas). Using kubectl we directly access to the Primary Pod and we use the Postgres official liberaries to apply restorations from an external volume. The external volume is manually mounted by editing the Primary's Statefulset.

Once the restoration is completed, we increase the number of instances of Postgres in the cluster with Kubegres YAML which then creates Replica Pods.

If I understand, you would like to automatise the process above by introducing a new field (restoreFromVolume) in the YAML?

[cslovell]

Hi Alex,

Yes - just thinking about how this task could be made simpler and automation would be valuable. Automating your approach would certainly work. Another way that came to mind just now was to use the new apiVersion "snapshot.storage.k8s.io/v1" which has the kind: "VolumeSnapshot". Restoring from a volume snapshot is simple in v1.20+, so this could be another good approach.

https://kubernetes.io/docs/concepts/storage/volume-snapshots/

Not sure the best way to do this - but perhaps others would like this feature? It would be a valuable way to make "bulletproof" database backups.

[alex-arica]

Thank you for your message. Sorry I could not reply earlier. I was away for a couple of weeks.

I agree that the support of kind: "VolumeSnapshot" would be very valuable. I am adding this feature in the backlog.

However, it may take sometimes before it is implemented because I am the only dev assigned to Kubegres from my company. Unless if another dev decides to help by sending a pull request for this feature, it will take time.

[weberc2]

It might be nice for the automated backups to allow for backups to object storage rather than assuming volume-based backups (for example, I pg_dumpall to S3).

EDIT:

Once the restoration is completed, we increase the number of instances of Postgres in the cluster with Kubegres YAML which then creates Replica Pods.

Since this is a new cluster, presumably it has a different name and thus you have to re-point your services to the new cluster (and then delete the old cluster, clean up its PVCs and PVs, etc)?

[gohjkc]

Hi, thank you for your kind feedback.

Currently the way we do restoration is manual: we create a new cluster of Postrgres with Kubegres with one Primary only (no replicas). Using kubectl we directly access to the Primary Pod and we use the Postgres official liberaries to apply restorations from an external volume. The external volume is manually mounted by editing the Primary's Statefulset.

Once the restoration is completed, we increase the number of instances of Postgres in the cluster with Kubegres YAML which then creates Replica Pods.

If I understand, you would like to automatise the process above by introducing a new field (restoreFromVolume) in the YAML?

Hi, using this thread instead of creating a new issue as I have queries related to this issue.

From your earlier reply, am I right to say that the only way I am able to access the backed up files is to create a new Pod and mount it with the PVC? Also, could you elaborate more on increasing the number of instances of Postgres with Kubegres YAML? Sorry if I'm asking something that has been asked before, I tried looking around but could not find anything.

Thanks in advance!