one icon indicating copy to clipboard operation
one copied to clipboard

new Backups: use DATASTORE/BRIDGE_LIST to provide a staging host(s)

Open atodorov-storpool opened this issue 2 years ago • 1 comments

Description Currently, the KVM host is used to prepare/stage the disk backups before handling them to the backup datastore MAD. When shared storage is used (NFS, Ceph, other SDS) it is possible to have not enough free space on the KVM host to dump the backups.

Use case Having a DATASTORE/BRIDGE_LIST the TM_MAD scripts could do the backups on a node from the list instead of the KVM host.

Interface Changes Utilise DATASTORE/BRIDGE_LIST attribute for the backup datastores. The prebackup/postbackup scripts should dump the backup files there instead of the VM home on the KVM node. The Backup datastore_mad scripts should use the bridge nodes too.

Additional options - add the hosts to the opennebula monitoring, so the backup scripts can select a backup_bridge_node that is up and had enough free space to handle( at least the sum of the allocated space by the VM disks)

Additional Context With shared storage, the hosts could have only small root disks for the host OS and the VM metadata files like the checkpoint, domain XML, etc. This way the backup scripts could become more "flexible".

In the case of Ceph, the "staging" node should be a Ceph client only and be used to dump/restore disk data to the storage.

Also, these backup nodes could be the actual backup servers. in that case, they could handle rsync/restic locally instead of accessing remotely. Yes, not the primary target for some of the definitions of backup DR, but it is a valid possibility.

Progress Status

  • [ ] Code committed
  • [ ] Testing - QA
  • [ ] Documentation (Release notes - resolved issues, compatibility, known issues)

atodorov-storpool avatar Dec 15 '22 14:12 atodorov-storpool

In other words such a bridge node is known as a bridge between the "Primary Storage" and the "Backup Storage"

atodorov-storpool avatar Dec 15 '22 14:12 atodorov-storpool