Create new Instance from VM backup

[abh1sar]

Doc PR: https://github.com/apache/cloudstack-documentation/pull/474

Description

This PR adds the ability to create a new Instance from a VM backup for Dummy, NAS and Veeam backup Providers. This will still work if the original Instance used to create the backup was expunged.

New API

• createVmFromBackup extends baseDeployVIrtualMachine
• new arguments : backupId, preserveip
• User can just specify the backupId and a new instance will be created by getting the VM and volumes' configuration stored in backup_details table.
• User can also choose to configure the details themselves using the parameters present in deployVIrtualMachine cmd
• If the original instance from which the backup was taken is deleted, user can choose to reuse the same IP addresses if they are available using the preserveip argument.

UI

• New action button in backups list view - 'Create Instance from Backup'
• User can choose to directly create the VM or configure VM parameters before creating it.

DB changes

• New backup_details table
  • to store vm config data such as serviceOfferingUuid, networkUuids, ipAddresses
  • to store volumes config data such as diskOfferingUuids, size, miniops and maxiops for custom disk offerings.

Other Changes

• Added new parameter Map datadisksdetails to deployVirtualMachine api to create multiple data volumes at the time of instance creation.

Plugins related changes

• NAS and Veeam plugin changes to restore backup to a new VM and miscellaneous fixes around handling backups of expunged Instances.

Other Miscellaneous changes

1. Add Name and description column to backup
2. Show Interval type in UI for backups
3. Support cephFS for NAS backup provider
4. Add Backup Storage Capacity Tracking and Alerts Infrastructure Backup Storage (used and available) is shown on the Capacity dashboard and tracks the alerts if supported by the plugin (Currently only NAS BnR plugin supports it)
5. Add Object Storage Capacity Tracking and Alerts Infrastructure Object Storage (allocated and available) is shown on the Capacity dashboard and tracked for alerts.

Types of changes

• [ ] Breaking change (fix or feature that would cause existing functionality to change)
• [x] New feature (non-breaking change which adds functionality)
• [ ] Bug fix (non-breaking change which fixes an issue)
• [ ] Enhancement (improves an existing feature and functionality)
• [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
• [ ] build/CI
• [ ] test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• [ ] Major
• [ ] Minor

Bug Severity

• [ ] BLOCKER
• [ ] Critical
• [ ] Major
• [ ] Minor
• [ ] Trivial

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

[codecov[bot]]

Codecov Report

:x: Patch coverage is 40.97744% with 1256 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 17.35%. Comparing base (70468a6) to head (226e82a). :warning: Report is 9 commits behind head on main.

Files with missing lines	Patch %	Lines
...loudstack/api/command/user/vm/BaseDeployVMCmd.java	47.91%	173 Missing and 39 partials :warning:
...rg/apache/cloudstack/backup/BackupManagerImpl.java	64.09%	111 Missing and 61 partials :warning:
.../src/main/java/com/cloud/vm/UserVmManagerImpl.java	44.25%	125 Missing and 40 partials :warning:
...ack/api/command/user/vm/CreateVMFromBackupCmd.java	6.15%	60 Missing and 1 partial :warning:
...rg/apache/cloudstack/backup/NASBackupProvider.java	34.40%	58 Missing and 3 partials :warning:
...rg/apache/cloudstack/backup/dao/BackupDaoImpl.java	21.81%	42 Missing and 1 partial :warning:
.../apache/cloudstack/backup/DummyBackupProvider.java	0.00%	42 Missing :warning:
...che/cloudstack/backup/NetworkerBackupProvider.java	0.00%	35 Missing :warning:
.../apache/cloudstack/backup/VeeamBackupProvider.java	5.88%	32 Missing :warning:
...a/org/apache/cloudstack/backup/BackupDetailVO.java	18.91%	29 Missing and 1 partial :warning:
... and 47 more

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #10140      +/-   ##
============================================
+ Coverage     17.17%   17.35%   +0.17%     
- Complexity    15010    15186     +176     
============================================
  Files          5869     5883      +14     
  Lines        521710   524302    +2592     
  Branches      63509    63962     +453     
============================================
+ Hits          89614    90974    +1360     
- Misses       422039   423051    +1012     
- Partials      10057    10277     +220     

Flag	Coverage Δ
uitests	3.63% <ø> (-0.13%)	:arrow_down:
unittests	18.38% <40.97%> (+0.22%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[leo79901]

Yes, we need this !

[abh1sar]

@blueorangutan package

[blueorangutan]

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11996

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[abh1sar]

@blueorangutan package

[blueorangutan]

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12039

[abh1sar]

@blueorangutan package

[blueorangutan]

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12063

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[abh1sar]

@blueorangutan package

[blueorangutan]

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12095

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[abh1sar]

one diskoffering - zone link edge case seems to be remaining for testing/changes Please see if we need to add some documentation for the feature

thanks @shwstppr I have fixed the disk-offering edge case and catch all exceptions plus some minor UI fixes. Also linked doc PR.

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[abh1sar]

@blueorangutan package

[blueorangutan]

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12734

[abh1sar]

@blueorangutan package

[blueorangutan]

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✖️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 12737

[abh1sar]

@blueorangutan package

[blueorangutan]

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.