[Veeam] Restore only a specified volume

[SadiJr]

Description

Using VMWare with Veeam on ACS, when restoring a volume from a backup, ACS creates a new VM from the backup, detaches all volumes, and attaches the selected volume to the original VM.

Veeam allows restoring only the selected volume, which is faster and more performant than the current approach. This PR aims to improve the volume restore process using this native option of Veeam.

Also, when restoring and attaching one volume to one VM, the target VM needs to be stopped and then started. However, the entire process is manual; even Veeam allows the VM to be started automatically if the restore is successful. A new parameter, startvm, has been added to the API restoreVolumeFromBackupAndAttachToVM, allowing operators to specify whether or not the VM should be started automatically when volume restoration is complete.

Types of changes

• [ ] Breaking change (fix or feature that would cause existing functionality to change)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Bug fix (non-breaking change which fixes an issue)
• [x] Enhancement (improves an existing feature and functionality)
• [ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• [ ] Major
• [x] Minor

How Has This Been Tested?

It was tested in a local lab:

1. I created a new VM and attached this VM to a Backup Offering;
2. I made some manual backups;
3. I restore some volume of this backups, without using the parameter startvm, and check if, in Veeam and vCenter, only the selected volume are restored, which results in success, but i had to start the VM manually;
4. Via cmk, I indicate the parameter startvm as true, and the VM is automatically started by Veeam when restore finishes.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 4.18%. Comparing base (8806e44) to head (5120402). Report is 58 commits behind head on main.

:exclamation: Current head 5120402 differs from pull request most recent head cda1c45

Please upload reports for the commit cda1c45 to get more accurate results.

:exclamation: There is a different number of reports uploaded between BASE (8806e44) and HEAD (5120402). Click for more details.

HEAD has 1 upload less than BASE

Flag	BASE (8806e44)	HEAD (5120402)
unittests	1	0

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #7221       +/-   ##
============================================
- Coverage     15.42%   4.18%   -11.24%     
============================================
  Files          5469     369     -5100     
  Lines        478292   30284   -448008     
  Branches      58134    5359    -52775     
============================================
- Hits          73775    1268    -72507     
+ Misses       396389   28872   -367517     
+ Partials       8128     144     -7984     

Flag	Coverage Δ
uitests	4.18% <ø> (ø)
unittests	?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[sonarqubecloud[bot]]

SonarCloud Quality Gate failed.   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
6 Code Smells

16.7% Coverage
0.0% Duplication

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[shwstppr]

@SadiJr can you please check the review comments?

[DaanHoogland]

@SadiJr are you still looking to get this merged?

[SadiJr]

@shwstppr @DaanHoogland Sorry for the delay, I will check the suggestions and respond/work on this PR

[shwstppr]

@blueorangutan package

[blueorangutan]

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 8008

[shwstppr]

@blueorangutan test

[blueorangutan]

@shwstppr a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian test result (tid-8536) Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7 Total time taken: 51064 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7221-t8536-kvm-centos7.zip Smoke tests completed. 121 look OK, 0 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

[weizhouapache]

@SadiJr can you please add some integration test cases ?

[weizhouapache]

this will power on the vm from veeam, looks like a out-of-band operation (the power state, host_id, etc are not managed by cloudstack).

@SadiJr I think we should avoid it. cc @DaanHoogland

[SadiJr]

@SadiJr can you please add some integration test cases ?

@weizhouapache sure, I'll try to add some integration tests

[weizhouapache]

I tested this PR with veeam 12. The integration test added in #8241 has some failures, some functionalities are broken by this .

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[JoaoJandre]

@blueorangutan package

[JoaoJandre]

@weizhouapache could we run the integration tests here so I can take a look at the errors you reported?

[weizhouapache]

@weizhouapache could we run the integration tests here so I can take a look at the errors you reported?

sure @blueorangutan test rocky8 vmware-80

[weizhouapache]

@weizhouapache could we run the integration tests here so I can take a look at the errors you reported?

@JoaoJandre #8241 requires a vmware/veeam environment. It will be skipped in the trillian test

[blueorangutan]

@weizhouapache a [SL] Trillian-Jenkins test job (rocky8 mgmt + vmware-80) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian test result (tid-10542) Environment: vmware-80 (x2), Advanced Networking with Mgmt server r8 Total time taken: 50658 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7221-t10542-vmware-80.zip Smoke tests completed. 118 look OK, 3 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_02_balanced_drs_algorithm	Error	422.24	test_cluster_drs.py
test_01_browser_migrate_template	Failure	5.47	test_image_store_object_migration.py
test_01_unmanage_vm_cycle	Error	128.04	test_vm_lifecycle_unmanage_import.py

[JoaoJandre]

@weizhouapache could we run the integration tests here so I can take a look at the errors you reported?

@JoaoJandre #8241 requires a vmware/veeam environment. It will be skipped in the trillian test

@weizhouapache Ok, I'll try to test this and see if everything still looks good after merging with main.

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[sureshanaparti]

hi @SadiJr please check and resolve any conflicts in the branch.