helm-charts icon indicating copy to clipboard operation
helm-charts copied to clipboard

Add Support for GCS and Azure backups

Open gunnarsundberg opened this issue 1 year ago • 9 comments

pgBackRest already supports backups to GCS buckets and Azure Storage. This PR removes the default S3 config to allow for alternate providers, adds a simple method of configuring a GCP service account key through secrets, and documentation for how to configure the Helm Chart to use S3, GCS, or Azure.

Issue: #272 and #389

gunnarsundberg avatar Jul 20 '22 00:07 gunnarsundberg

Question: doesn't #389 affect this use case at all?

agronholm avatar Jul 21 '22 18:07 agronholm

Yes it does. #390 would fix this, but adds Azure documentation through comments in values.yaml and does not store the Azure key through secrets. It may be confusing to have GCS and S3 documented in the admin guide, with Azure documented in values.yaml and a different key config method from GCS.

gunnarsundberg avatar Jul 21 '22 19:07 gunnarsundberg

Yes, it would be preferable to get these changes coordinated (I am an Azure user myself).

agronholm avatar Jul 21 '22 19:07 agronholm

Would you suggest I wait to see how 390 shakes out, or just go ahead and add Azure to this PR? Seems like 390 is hung up on something unrelated.

gunnarsundberg avatar Jul 21 '22 19:07 gunnarsundberg

You should probably coordinate with its author. I'm just a concerned citizen :)

agronholm avatar Jul 21 '22 19:07 agronholm

@gunnarsundberg Do you intend to finish this PR? I kinda need it. Let me know if I can help in any way, or if you'd like me to finish it myself.

agronholm avatar Sep 01 '22 08:09 agronholm

@agronholm I intend to finish it but have been quite busy. If you'd like it finished in the immediate future I'm cool with you doing it. If not I'll pick it back up at a later time 🙂

gunnarsundberg avatar Sep 12 '22 22:09 gunnarsundberg

@agronholm I intend to finish it but have been quite busy. If you'd like it finished in the immediate future I'm cool with you doing it. If not I'll pick it back up at a later time slightly_smiling_face

The easiest way would be if you granted me push access to your fork. Otherwise I could just copy everything to my fork and make a new PR, but I see that as unnecessary hassle.

agronholm avatar Sep 13 '22 14:09 agronholm

@paulfantom would you mind approving the test run? I believe I've fixed the issue.

agronholm avatar Sep 14 '22 08:09 agronholm

Is there anyway that we could rerun the tests to see the failure - would be nice to have this rolled into the main branch.

DanChugani avatar Dec 14 '22 02:12 DanChugani

Would also love to have this merged 🤞. We want to go to production soon on GKE and need support for backups to GCS.

shooit avatar Jan 02 '23 14:01 shooit

Would also love to have this merged crossed_fingers. We want to go to production soon on GKE and need support for backups to GCS.

We're probably moving to the StackGres operator for deploying TimescaleDB.

agronholm avatar Jan 02 '23 17:01 agronholm

@gunnarsundberg can you fix the linting issues? Looks like you just need to bump the helm chart version. 0.30.0 -> 0.31.0 should be fine.

Linting charts...

------------------------------------------------------------------------------------------------------------------------
 Charts to be processed:
------------------------------------------------------------------------------------------------------------------------
 timescaledb-single => (version: "0.30.0", path: "charts/timescaledb-single")
------------------------------------------------------------------------------------------------------------------------

Linting chart "timescaledb-single => (version: \"0.30.0\", path: \"charts/timescaledb-single\")"
Checking chart "timescaledb-single => (version: \"0.30.0\", path: \"charts/timescaledb-single\")" for a version bump...
Old chart version: 0.30.0
Error: failed linting charts: failed processing charts
New chart version: 0.30.0
------------------------------------------------------------------------------------------------------------------------
 ✖︎ timescaledb-single => (version: "0.30.0", path: "charts/timescaledb-single") > chart version not ok. Needs a version bump! 
make: *** [Makefile:[16](https://github.com/timescale/helm-charts/actions/runs/3955995463/jobs/6776051445#step:6:17): lint] Error 1
------------------------------------------------------------------------------------------------------------------------
failed linting charts: failed processing charts
Error: Process completed with exit code 2.

nhudson avatar Jan 19 '23 08:01 nhudson

@nhudson done!

gunnarsundberg avatar Jan 19 '23 16:01 gunnarsundberg

@gunnarsundberg looks like there is a lint issue and if you don't mind rebasing again?

nhudson avatar Jan 23 '23 14:01 nhudson