cloudstack icon indicating copy to clipboard operation
cloudstack copied to clipboard

KVM: Option to deploy a VM with existing volume/snapshot

Open slavkap opened this issue 9 months ago • 48 comments

Description

This PR allows the users to deploy virtual machines with an existing volume or a snapshot. For now it is limited only for KVM hypervisors and Zone-wide storage pools

Types of changes

  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [X] New feature (non-breaking change which adds functionality)
  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [ ] Enhancement (improves an existing feature and functionality)
  • [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
  • [ ] build/CI
  • [ ] test (unit or integration test code)

Feature/Enhancement Scale

  • [X] Major
  • [ ] Minor

How Has This Been Tested?

Environment OS - Rocky Linux 8.9 2 Zones with one agent host on each hypervisor KVM

Tested manually and with smoke tests with StorPool and NFS as a primary storage

slavkap avatar Mar 05 '25 11:03 slavkap

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 3.90%. Comparing base (16c60c7) to head (b36e05b). :warning: Report is 186 commits behind head on main.

:exclamation: There is a different number of reports uploaded between BASE (16c60c7) and HEAD (b36e05b). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (16c60c7) HEAD (b36e05b)
unittests 1 0
Additional details and impacted files
@@              Coverage Diff              @@
##               main   #10503       +/-   ##
=============================================
- Coverage     16.57%    3.90%   -12.68%     
=============================================
  Files          5743      414     -5329     
  Lines        510494    33746   -476748     
  Branches      62075     6075    -56000     
=============================================
- Hits          84617     1317    -83300     
+ Misses       416415    32273   -384142     
+ Partials       9462      156     -9306     
Flag Coverage Δ
uitests 3.90% <ø> (ø)
unittests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codecov[bot] avatar Mar 05 '25 11:03 codecov[bot]

@blueorangutan package

DaanHoogland avatar Mar 06 '25 10:03 DaanHoogland

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Mar 06 '25 10:03 blueorangutan

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12681

blueorangutan avatar Mar 06 '25 11:03 blueorangutan

@blueorangutan test

DaanHoogland avatar Mar 06 '25 12:03 DaanHoogland

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan avatar Mar 06 '25 12:03 blueorangutan

[SF] Trillian test result (tid-12598) Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8 Total time taken: 63251 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10503-t12598-kvm-ol8.zip Smoke tests completed. 140 look OK, 2 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup Error 1520.36 test_network.py
test_01_deploy_vm_with_existing_volume Error 7.34 test_vm_lifecycle_with_snapshot_or_volume.py
test_02_deploy_vm_with_existing_snapshot Error 0.06 test_vm_lifecycle_with_snapshot_or_volume.py
test_03_deploy_vm_with_existing_volume_deleted_template Failure 861.05 test_vm_lifecycle_with_snapshot_or_volume.py
test_04_deploy_vm_with_existing_snapshot_deleted_template Failure 862.76 test_vm_lifecycle_with_snapshot_or_volume.py
test_05_deploy_vm_with_existing_snapshot_deleted_volume Failure 847.20 test_vm_lifecycle_with_snapshot_or_volume.py

blueorangutan avatar Mar 07 '25 06:03 blueorangutan

@blueorangutan package

DaanHoogland avatar Mar 10 '25 14:03 DaanHoogland

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Mar 10 '25 15:03 blueorangutan

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12727

blueorangutan avatar Mar 10 '25 16:03 blueorangutan

@blueorangutan test

DaanHoogland avatar Mar 11 '25 07:03 DaanHoogland

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan avatar Mar 11 '25 07:03 blueorangutan

[SF] Trillian test result (tid-12645) Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8 Total time taken: 61921 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10503-t12645-kvm-ol8.zip Smoke tests completed. 140 look OK, 2 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup Error 1520.50 test_network.py
test_01_deploy_vm_with_existing_volume Error 0.00 test_vm_lifecycle_with_snapshot_or_volume.py
test_02_deploy_vm_with_existing_snapshot Error 0.00 test_vm_lifecycle_with_snapshot_or_volume.py
test_03_deploy_vm_with_existing_volume_deleted_template Error 0.00 test_vm_lifecycle_with_snapshot_or_volume.py
test_04_deploy_vm_with_existing_snapshot_deleted_template Error 0.00 test_vm_lifecycle_with_snapshot_or_volume.py
test_05_deploy_vm_with_existing_snapshot_deleted_volume Error 0.00 test_vm_lifecycle_with_snapshot_or_volume.py

blueorangutan avatar Mar 12 '25 01:03 blueorangutan

@slavkap the tests fail because the self doesn't have a snapshot that was supposed to be created in the class setup method. Is this a timing issue?

DaanHoogland avatar Mar 12 '25 10:03 DaanHoogland

@blueorangutan package

slavkap avatar Mar 12 '25 11:03 slavkap

@slavkap a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Mar 12 '25 11:03 blueorangutan

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12756

blueorangutan avatar Mar 12 '25 12:03 blueorangutan

@blueorangutan test

slavkap avatar Mar 12 '25 19:03 slavkap

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

github-actions[bot] avatar Apr 28 '25 08:04 github-actions[bot]

@blueorangutan package

DaanHoogland avatar Apr 30 '25 07:04 DaanHoogland

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Apr 30 '25 07:04 blueorangutan

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13223

blueorangutan avatar Apr 30 '25 09:04 blueorangutan

@blueorangutan test

DaanHoogland avatar Apr 30 '25 09:04 DaanHoogland

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan avatar Apr 30 '25 09:04 blueorangutan

[SF] Trillian test result (tid-13186) Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8 Total time taken: 54496 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10503-t13186-kvm-ol8.zip Smoke tests completed. 141 look OK, 1 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_02_restore_vm_strict_tags_failure Failure 60.49 test_vm_strict_host_tags.py
test_02_scale_vm_strict_tags_failure Failure 58.64 test_vm_strict_host_tags.py
test_06_deploy_vm_on_any_host_with_strict_tags_failure Failure 5.67 test_vm_strict_host_tags.py

blueorangutan avatar May 01 '25 01:05 blueorangutan

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

github-actions[bot] avatar May 12 '25 14:05 github-actions[bot]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

github-actions[bot] avatar May 16 '25 16:05 github-actions[bot]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

github-actions[bot] avatar May 22 '25 06:05 github-actions[bot]

Tested this with Linstor primary storage, works there too.

The only thing I noticed that on Linstor the vm name and volume name is missing. I guess the volume is created in the agent adaptor code and Linstor gets this information out of the KVMPhysicalDisk(dispName,vmName) object, and it is probably not set/copied there?

rp- avatar May 22 '25 11:05 rp-

@blueorangutan package

weizhouapache avatar May 28 '25 09:05 weizhouapache