apache
backup: simple NAS backup plugin for KVM (experimental)
This is an experimental simple NAS backup plugin for KVM which may be later expanded for other hypervisors. This backup plugin aims to use shared NAS storage on KVM hosts such as NFS or CephFS, which is used to backup fully cloned VMs for backup & restore operations. This may not be as efficient and performant as some of the other B&R providers, but maybe useful for some KVM environments.
Design & Implementation follows the networker B&R plugin, which is simply:
- Implement B&R plugin interfaces
- Use cmd-answer pattern to execute backup and restore operations on KVM host when VM is running (or needs to be restored) - instead of a B&R API client, relies on answers from KVM agent which executes the operations
- Backups are full VM domain snapshots, copied to a VM-specific folders on a NAS target (NFS or CephFS WIP) along with a domain XML
- Backup uses libvirt feature: https://libvirt.org/kbase/live_full_disk_backup.html orchestrated via virsh/bash script (nasbackup.sh) as libvirt-java lacks the bindings
Doc PR https://github.com/apache/cloudstack-documentation/pull/429
Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
- [ ] build/CI
- [ ] test (unit or integration test code)
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
- [x] Major
- [ ] Minor
Screenshots
Each backup offering map to a Backup repository that takes in backup storage details (address/url, credentials/mount options):
Rest is same as B&R feature works; we assign such offering to a VM and backup once completed shows up in the list:
Codecov Report
Attention: Patch coverage is 0.62696% with 951 lines in your changes missing coverage. Please review.
Project coverage is 15.61%. Comparing base (
605534b) to head (9efdb57). Report is 2 commits behind head on main.
Additional details and impacted files
@@ Coverage Diff @@
## main #9451 +/- ##
============================================
+ Coverage 4.12% 15.61% +11.49%
- Complexity 0 12118 +12118
============================================
Files 383 5555 +5172
Lines 31354 486283 +454929
Branches 5551 61315 +55764
============================================
+ Hits 1293 75945 +74652
- Misses 29915 401966 +372051
- Partials 146 8372 +8226
| Flag | Coverage Δ | |
|---|---|---|
| uitests | 4.12% <ø> (+<0.01%) |
:arrow_up: |
| unittests | 16.40% <0.62%> (?) |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
![codecov[bot] avatar](https://avatars.githubusercontent.com/in/254?v=4 "Published by a pub.dev verified publisher"){kind=small}
Update: take backup & delete backup implemented for NFS; TBD/TODO more operations.
@rohityadavcloud a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10548
Tested NFS, CephFS & samba, works when configuration is correctly configured; for example;
mysql> select * from backup_repository\G;
*************************** 1. row ***************************
id: 1
uuid: d3b911d4-50a6-11ef-bc8b-525400f93984
name: NFS-backup
zone_id: 1
provider: nas
type: nfs
address: 172.20.0.1:/export/testing/qa1-cs420-kvm-1cdf7971-kvm/backup
mount_opts:
used_bytes: NULL
capacity_bytes: NULL
created: 2024-08-02 08:11:31
removed: NULL
*************************** 2. row ***************************
id: 2
uuid: 09c2efd1-50c4-11ef-bc8b-525400f93984
name: Ceph-backup
zone_id: 1
provider: nas
type: ceph
address: 10.10.1.10,10.10.1.11,10.10.1.12:/backup
mount_opts: name=rohit,secret=XXXX==
used_bytes: NULL
capacity_bytes: NULL
created: 2024-08-02 11:40:37
removed: NULL
*************************** 3. row ***************************
id: 3
uuid: 876b1da4-50c6-11ef-bc8b-525400f93984
name: Samba-backup
zone_id: 1
provider: nas
type: cifs
address: //192.168.1.10/backup
mount_opts: vers=3.0,username=rohit,password=XXXX
used_bytes: NULL
capacity_bytes: NULL
created: 2024-08-02 11:58:27
removed: NULL
3 rows in set (0.00 sec)
Shared storage mount/umount tested:
- EL8 & EL9: NFS & CephFS works (ceph would require rbd utils, otherwise uses Linux kernel's cephfs driver)
- Ubuntu: NFS, CephFS & CIFS/SMB3 works (the latter two need cifs-utils & ceph utils)
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 10579
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 10580
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10583
@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10621
@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10624
Update cc @DaanHoogland @sureshanaparti @shwstppr @rajujith @NuxRo the experimental plugin is ready for review & testing, pending a bit of cleanup, maybe tests & doc work. Tested following cases:
- Backup of VM with root disk & VM with root & data disks
- Restore of backup of VM with root disk & VM with root & data disks
- Deleting backups of VM with root disk & VM with root & data disks
- Restore a volume of a backup of VM
Case WIP: restoring VM using backup for a VM which is expunged
@blueorangutan package
@rohityadavcloud a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
@Pearl1594 do we have more (internal?) details on the design of this one? Looks really interesting.
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10847
There isn't a design doc or spec, I'll submit a doc PR which can be used for notes. The plugin simply uses existing B&R framework and follows on the footsteps of networker plugin, so its well isolated as yet another plugin - all supported B&R functionality work as-is, doc PR will make it further clear.
@blueorangutan test
@rohityadavcloud a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests