cloudstack
cloudstack copied to clipboard
apache
Change storage pool scope from Cluster to Zone and vise versa
Description
This PR adds the functionality to change the scope of the Primary storage from zone to cluster scope and vice versa.
Feature is tested and supported for Hypervisors : KVM and VMware (Simulator will also work) Protocol : NFS and Ceph, and Storage Provider : Default Primary
Although the supported configuration is mentioned above, the root admin can use this feature for other configurations as well, but proper warning would be given that officially this feature is only tested for a limited set of hypervisor-storage configuration. And other configurations might not work and need some extra steps. We will give this information in the UI prompt as well as Api docs. Apache CloudStack documentation will also get a mention on what hypervisors/storage is supported.
Documentation PR : https://github.com/apache/cloudstack-documentation/pull/395
The option for scope change is provided as a new action button to root admin once a Primary storage has been disabled.
A new api changeStoragePoolScope handles the execution after doing all relevant checks. It makes DB changes and also connects to the hypervisor hosts to add or remove the storage pool as required. It adds the storage pool to the applicable hosts when the scope is changing from Cluster to Zone. And it removes the storage pool from the hosts when the scope is changing from Zone to Cluster. Management server or the Cloudstack agents don’t need to be restarted for this.
Scope change from Zone to Cluster cannot be done if there are volumes associated with running-VMs belonging to other clusters. Api will throw an error in this case.
Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
- [ ] build/CI
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
- [ ] Major
- [x] Minor
Bug Severity
- [ ] BLOCKER
- [ ] Critical
- [ ] Major
- [ ] Minor
- [ ] Trivial
Screenshots (if appropriate):
How Has This Been Tested?
KVM + NFS
Cluster to Zone
- Verified that Scope is changing correctly. Hosts in other clusters have the storage pool mounted.
- New instance on other clusters are able to use the storage pool
- Offline a host in Cluster C2. Change the scope from C1 to Zone. Online the host and verify that the storage pool gets mounted on the host.
Zone to Cluster
- Verified that Scope is changing correctly. Hosts in other clusters don't have the storage pool mounted.
- Offline a host in Cluster C2. Change the scope from Zone to Cluster C1. Online the host and verify that the storage pool gets gets removed from the host.
- Deploy a VM in Cluster C2 having a volume in the Zone wide storage pool. Change scope from Zone to C1 fails as we have a volume from C2 in the storage pool.
- Repeat of # 2 but the VM is in stopped state. This time scope change from Zone to C1 succeeds. Restart of the VM succeeds in C1 as the storage pool is now present in C1.
VMware + NFS
Tested basic functionality.
How did you try to break this feature and the system with this change?
@rohityadavcloud a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9138
@andrijapanicsb a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests
Codecov Report
Attention: Patch coverage is 48.17927% with 185 lines in your changes missing coverage. Please review.
Project coverage is 15.77%. Comparing base (
814c8b6) to head (1b698c1).
Additional details and impacted files
@@ Coverage Diff @@
## 4.19 #8875 +/- ##
============================================
+ Coverage 15.01% 15.77% +0.76%
- Complexity 11086 11121 +35
============================================
Files 5393 5033 -360
Lines 471529 442426 -29103
Branches 59743 53754 -5989
============================================
- Hits 70788 69782 -1006
+ Misses 392854 364872 -27982
+ Partials 7887 7772 -115
| Flag | Coverage Δ | |
|---|---|---|
| uitests | ? |
|
| unittests | 15.77% <48.17%> (+0.04%) |
:arrow_up: |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
[SF] Trillian test result (tid-9682) Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7 Total time taken: 52569 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8875-t9682-kvm-centos7.zip Smoke tests completed. 127 look OK, 2 have errors, 0 did not run Only failed and skipped tests results shown below:
| Test | Result | Time (s) | Test File |
|---|---|---|---|
| test_01_primary_storage_scope_change | Error |
0.06 | test_primary_storage.py |
| test_02_trigger_shutdown | Failure |
341.66 | test_safe_shutdown.py |
In a discussion with @rohityadavcloud and @andrijapanicsb it was decided to not gate this feature for other hypervisor and storage types. We will give the root admin the option to change the scope, but by giving enough warning that officially this feature is only tested for a limited set of hypervisor-storage configuration. And other configurations might not work and need some extra steps.
We will give this information in the UI prompt as well as Api docs. Apache CloudStack documentation will also get a mention on what hypervisors/storage is supported.
So, I have removed the checks in UI and StorageManagerImpl, and added proper warning.
Only check present is for hypervisorType which is same as the hypervisor types currently supported for Zone wide primary storage.
@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9203
@weizhouapache a [SL] Trillian-Jenkins test job (rocky8 mgmt + kvm-rocky8) has been kicked to run smoke tests
@weizhouapache a [SL] Trillian-Jenkins test job (rocky8 mgmt + kvm-rocky8) has been kicked to run smoke tests
[SF] Trillian test result (tid-9766) Environment: kvm-rocky8 (x2), Advanced Networking with Mgmt server r8 Total time taken: 51815 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8875-t9766-kvm-rocky8.zip Smoke tests completed. 129 look OK, 0 have errors, 0 did not run Only failed and skipped tests results shown below:
| Test | Result | Time (s) | Test File |
|---|
@abh1sar can you
- pull or merge the latest 4.19 on your PR branch
- resolve all review comments you've addressed
- address any other outstanding remark(s)
- ask for a re-review from reviewers whose comments you've addressed thnx
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
@shwstppr @weizhouapache @DaanHoogland I have addressed the review comments. Please check.
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Packaging result [SF]: ✖️ el7 ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 9420
@shwstppr @weizhouapache I have added some UT to increase coverage. Please check.
@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9474