WIP: feat: add dedicated backup role

[pat-s]

fix #916

• move default backups vars from cloud-resources to backup role
• Execute backup role tasks when backup_provider is set
• Execute backup role when pgbackrest_install or wal_g_install are true

[vitabaks]

@pat-s Why did you move the tasks of creating backup buckets to a separate role, this "backup" role name is not quite suitable since this is done by the pgbackrest or wal-g roles.

What about this approach? https://github.com/vitabaks/autobase/issues/916#issuecomment-2720066513 (use conditions instead of moving tasks to a separate role)

Or we could rename it to "bucket". And execute auto_conf tasks if the backup_provider and pgbackrest_auto_conf / wal_g_auto_conf variables are filled in.

[pat-s]

Why did you move the tasks of creating backup buckets to a separate role, this "backup" role name is not quite suitable since this is done by the pgbackrest or wal-g roles.

It made sense to me to bundle everything related to backups into this dedicated role. I don't see why the backup buckets should be created in another role. Feel free to re-arrange, though!

What about this approach? https://github.com/vitabaks/autobase/issues/916#issuecomment-2720066513 (use conditions instead of moving tasks to a separate role)

Saw it in the process but didn't like it so much. Also see the previous comment here. "backup" is an essential task which imo deserves its own role. Especially as there are multiple tools that could be used to execute the backup, it made sense to me to bundle all tool-agnostic actions into this role. But in the end it's up to you how to structure it :) Feel free to re-arrange.

I currently have a working state using the structure for pgbackrest for both the deploy and configure playbooks. Didn't test it with the other options though.

Or we could rename it to "bucket".

The role you mean? I think "backup" is much more generic, especially if there should be other means of taking backups than s3. Given that s3 is also the protocol name and used as a general term these days, keeping it as the "type" identifier for bucket/s3 backups sounds reasonable.

[vitabaks]

It made sense to me to bundle everything related to backups into this dedicated role.

Then I think the current roles such as pgbackrest and wal-g should become tasks in the backup role.

[vitabaks]

Then I think the current roles such as pgbackrest and wal-g should become tasks in the backup role.

@pat-s Do you have time to do this?

[pat-s]

Not right now. Feel free to continue.

[pat-s]

@vitabaks Are you planning to work on this in the near future or include it at all? An S3-backup with a free provider choice is the last part right now which I am missing before I can move back to the upstream module of yours again.

I don't really care as a user if this is a dedicated role or split across existing roles. In the end, I want to be able to set something like this in my config to use S3 backups via pgbackrest:

pgbackrest_repo_type: 's3'

s3_backup_provider: <cloud provider>
<cloud_provider>_object_storage_XYZ: 

[vitabaks]

@pat-s Regarding the new “backup” role: it lets you provision your own servers (e.g. via Terraform) while Autobase handles software and optionally creates a single cloud resource like an S3 bucket.

However, if you’re already provisioning servers, why not provision the bucket too and configure pgBackRest using our docs?

Currently we support two modes:

1. BYOC — Autobase provisions Cloud resources & configures software
2. BYOM — Autobase manages software on your own machones.

If you’d like a hybrid mode (BYOM + S3 Bucket), we can do it. I’ll take a closer look shortly, just after we release version 2.3.0.

[pat-s]

I can't quite follow. What does this have to do with provisioning servers and what Autobase is doing?

I use BYOM, yes. And what I essentially need is #979. I want to back up to Hetzner S3 via pgbackrest. This PR here is just another stab which reorganizes backup functionality within autobase as a whole, as you didn't seem to like the approach in #979, due to what I assume being in the auto_conf role?

configure pgBackRest using our docs?

Did I overlook something and this is already possible? Looking at the code, I had the impression it isn't and I had to apply the mentioned patches.

[vitabaks]

Did I overlook something and this is already possible?

It possible :)

https://autobase.tech/docs/management/backup#command-line

[pat-s]

Ah, apologizes then! Didn't see that. I only checked the code and tried to patch it to make it work with the options in auto_conf etc. Tested the "official" method now and it works.

Anyhow, a dedicated "backup" role might be beneficial WRT to overall organization etc ;) But no pressure then (anymore) from my side 😄️