cloudstack icon indicating copy to clipboard operation
cloudstack copied to clipboard

linstor: Support VM-Instance Disk snapshots

Open rp- opened this issue 1 year ago • 15 comments

Description

This adds VM-Instance disk snapshot support for Linstor primary storage. Instance snapshots are stored on the used Linstor storage pool backend and can be converted into regular volume snapshots and also reverted.

Instance VM snapshots are not fully atomic but with the create multi snapshot feature as good as it gets. Snapshots are done over multiple volumes in the same devicemanager run.

Types of changes

  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [x] Enhancement (improves an existing feature and functionality)
  • [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
  • [ ] build/CI

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • [ ] Major
  • [x] Minor

Screenshots (if appropriate):

How Has This Been Tested?

Tested on hyper and non hyper-converged cluster, with LVM and ZFS as Linstor primary storage backend.

  • Making a VM-Snapshot with multiple disks
  • Snapshot from VM-Snapshot disk
  • Revert to VM-Snapshot

rp- avatar Mar 17 '24 20:03 rp-

Codecov Report

Attention: Patch coverage is 0% with 209 lines in your changes are missing coverage. Please review.

Project coverage is 30.93%. Comparing base (6c6023b) to head (151fde5).

Files Patch % Lines
...ck/storage/snapshot/LinstorVMSnapshotStrategy.java 0.00% 185 Missing :warning:
...tore/driver/LinstorPrimaryDataStoreDriverImpl.java 0.00% 22 Missing :warning:
...cloudstack/storage/datastore/util/LinstorUtil.java 0.00% 2 Missing :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##               4.19    #8796      +/-   ##
============================================
+ Coverage     30.13%   30.93%   +0.79%     
- Complexity    33072    34326    +1254     
============================================
  Files          5355     5356       +1     
  Lines        376690   376886     +196     
  Branches      54815    54842      +27     
============================================
+ Hits         113533   116595    +3062     
+ Misses       248185   244941    -3244     
- Partials      14972    15350     +378     
Flag Coverage Δ
simulator-marvin-tests 24.77% <0.00%> (+1.08%) :arrow_up:
uitests 4.38% <ø> (ø)
unit-tests 16.56% <0.00%> (-0.01%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Mar 18 '24 15:03 codecov[bot]

@blueorangutan package

DaanHoogland avatar Mar 29 '24 16:03 DaanHoogland

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Mar 29 '24 16:03 blueorangutan

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9100

blueorangutan avatar Mar 29 '24 17:03 blueorangutan

@blueorangutan test alma9 kvm-alma9

DaanHoogland avatar Apr 02 '24 09:04 DaanHoogland

@DaanHoogland a [SL] Trillian-Jenkins test job (alma9 mgmt + kvm-alma9) has been kicked to run smoke tests

blueorangutan avatar Apr 02 '24 09:04 blueorangutan

@blueorangutan package

DaanHoogland avatar Apr 02 '24 15:04 DaanHoogland

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Apr 02 '24 15:04 blueorangutan

Codecov Report

Attention: Patch coverage is 0% with 213 lines in your changes are missing coverage. Please review.

Project coverage is 14.95%. Comparing base (87b55af) to head (393afc0).

Files Patch % Lines
...ck/storage/snapshot/LinstorVMSnapshotStrategy.java 0.00% 189 Missing :warning:
...tore/driver/LinstorPrimaryDataStoreDriverImpl.java 0.00% 22 Missing :warning:
...cloudstack/storage/datastore/util/LinstorUtil.java 0.00% 2 Missing :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##               4.19    #8796      +/-   ##
============================================
- Coverage     14.95%   14.95%   -0.01%     
- Complexity    10986    10991       +5     
============================================
  Files          5373     5374       +1     
  Lines        469175   469375     +200     
  Branches      57254    60913    +3659     
============================================
+ Hits          70179    70195      +16     
- Misses       391226   391406     +180     
- Partials       7770     7774       +4     
Flag Coverage Δ
uitests 4.30% <ø> (ø)
unittests 15.66% <0.00%> (-0.01%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Apr 02 '24 16:04 codecov-commenter

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 9134

blueorangutan avatar Apr 02 '24 17:04 blueorangutan

[SF] Trillian test result (tid-9657) Environment: kvm-alma9 (x2), Advanced Networking with Mgmt server a9 Total time taken: 52947 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8796-t9657-kvm-alma9.zip Smoke tests completed. 128 look OK, 1 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_hostha_enable_ha_when_host_disabled Error 2.37 test_hostha_kvm.py
test_hostha_enable_ha_when_host_in_maintenance Error 301.38 test_hostha_kvm.py

blueorangutan avatar Apr 03 '24 01:04 blueorangutan

@blueorangutan test alma9 kvm-alma9

DaanHoogland avatar Apr 03 '24 08:04 DaanHoogland

@DaanHoogland a [SL] Trillian-Jenkins test job (alma9 mgmt + kvm-alma9) has been kicked to run smoke tests

blueorangutan avatar Apr 03 '24 08:04 blueorangutan

[SF] Trillian test result (tid-9678) Environment: kvm-alma9 (x2), Advanced Networking with Mgmt server a9 Total time taken: 48048 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8796-t9678-kvm-alma9.zip Smoke tests completed. 128 look OK, 1 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_01_secure_vm_migration Error 182.30 test_vm_life_cycle.py

blueorangutan avatar Apr 03 '24 22:04 blueorangutan

@sureshanaparti Do you have time for a review?

rp- avatar May 06 '24 10:05 rp-

updated 3rd party library by a patch version and rebased

rp- avatar May 31 '24 13:05 rp-

@DaanHoogland Can I get this added to the 4.19.1 milestone, this PR is now sitting around for a few months now...

rp- avatar Jun 11 '24 09:06 rp-

@DaanHoogland Can I get this added to the 4.19.1 milestone, this PR is now sitting around for a few months now...

You are free to do that yourself (as a committer) but I just did. @sureshanaparti will surely look at it ;)

@slavkap is this alright by you now?

DaanHoogland avatar Jun 12 '24 07:06 DaanHoogland

@DaanHoogland, yes, the code LGTM.

Instance VM snapshots are not fully atomic but with the create multi snapshot feature as good as it gets.

@rp-, a suggestion here - you can add a configuration setting with which you can enable to freeze the VM (if the users want the snapshots to be consistent)

I'll keep that in mind, but wouldn't want to change this PR now to much, before the 4.19.1 release. Also, I'm on vacation starting next week.

rp- avatar Jun 13 '24 06:06 rp-

@blueorangutan package

sureshanaparti avatar Jun 13 '24 07:06 sureshanaparti

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Jun 13 '24 07:06 blueorangutan

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9918

blueorangutan avatar Jun 13 '24 08:06 blueorangutan

Merging this, based on reviews & test results.

sureshanaparti avatar Jun 13 '24 09:06 sureshanaparti