apache
PowerFlex/ScaleIO MDM and host SDC connection enhancements
Description
This PR enhances the PowerFlex/ScaleIO MDM and host SDC connections, includes the following changes.
-
Introduced timeout configuration 'powerflex.mdm.change.apply.timeout.ms' (Default value: 1000 ms) at zone scope, to wait after MDM changes made on Host with ScaleIO SDC. Also, Changes to apply timeout after making MDM changes for ScaleIO in prepare and unprepare logic.
-
Introduced configuration flag 'powerflex.block.sdc.unprepare' (Default is false) at zone scope, to enable/disable blocking unprepare ScaleIO SDC connection when SDC client restart required and there are volumes attached to the Host. Added validation to fail Host disconnect from Storage Pool if there are Volumes attached and SDC client MDM removal requires scini service to be restarted.
-
Introduced configuration flag 'powerflex.mdm.validate.on.connect' (Default is false) at zone scope, to enable/disable validation of MDM addresses on Host from configuration file and from memory (using CLI) matches or not.
-
Added detection of MDM removal support via CLI. If MDM removal support via CLI supported then use CLI, Otherwise fall back to edit drv_cfg.txt and restart scini as earlier. Tested with /opt/emc/scaleio/sdc/bin/drv_cfg --version: DellEMC PowerFlex Version: R3_6.4000.124, with cmd: /opt/emc/scaleio/sdc/bin/drv_cfg --remove_mdm.
Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
- [ ] build/CI
- [ ] test (unit or integration test code)
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
- [x] Major
- [ ] Minor
Bug Severity
- [ ] BLOCKER
- [ ] Critical
- [ ] Major
- [ ] Minor
- [ ] Trivial
Screenshots (if appropriate):
How Has This Been Tested?
Tested the new settings, and PowerFlex SDC connections (MDM add/remove) with VM & Volume operations.
How did you try to break this feature and the system with this change?
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Codecov Report
:x: Patch coverage is 26.66667% with 242 lines in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 16.58%. Comparing base (be22bfe) to head (7433231).
:warning: Report is 185 commits behind head on main.
Additional details and impacted files
@@ Coverage Diff @@
## main #11047 +/- ##
==========================================
Coverage 16.57% 16.58%
- Complexity 13971 13988 +17
==========================================
Files 5743 5746 +3
Lines 510648 511103 +455
Branches 62105 62170 +65
==========================================
+ Hits 84641 84754 +113
- Misses 416534 416863 +329
- Partials 9473 9486 +13
| Flag | Coverage Δ | |
|---|---|---|
| uitests | 3.91% <ø> (+<0.01%) |
:arrow_up: |
| unittests | 17.47% <26.66%> (+<0.01%) |
:arrow_up: |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
- :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
![codecov[bot] avatar](https://avatars.githubusercontent.com/in/254?v=4 "Published by a pub.dev verified publisher"){kind=small}
Packaging result [SF]: ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 13811
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13812
@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
@harikrishna-patnala a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
[SF] Trillian test result (tid-13586) Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8 Total time taken: 69826 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11047-t13586-kvm-ol8.zip Smoke tests completed. 136 look OK, 5 have errors, 0 did not run Only failed and skipped tests results shown below:
| Test | Result | Time (s) | Test File |
|---|---|---|---|
| ContextSuite context=TestAccounts>:setup | Error |
0.00 | test_accounts.py |
| ContextSuite context=TestAddVmToSubDomain>:setup | Error |
0.00 | test_accounts.py |
| test_DeleteDomain | Error |
15.92 | test_accounts.py |
| test_forceDeleteDomain | Failure |
15.66 | test_accounts.py |
| ContextSuite context=TestRemoveUserFromAccount>:setup | Error |
16.38 | test_accounts.py |
| ContextSuite context=TestTemplateHierarchy>:setup | Error |
1537.55 | test_accounts.py |
| ContextSuite context=TestDeployVmWithAffinityGroup>:setup | Error |
0.00 | test_affinity_groups_projects.py |
| ContextSuite context=TestAnnotations>:setup | Error |
0.00 | test_annotations.py |
| ContextSuite context=TestAsyncJob>:setup | Error |
0.00 | test_async_job.py |
| ContextSuite context=TestClusterDRS>:setup | Error |
0.00 | test_cluster_drs.py |
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13907
@sureshanaparti as these are general bugfix/improvements for powerflex storage, should these be also applicable for 4.20 branch (esp these changes are scaleio/powerflex storage plugin related with no DB changes)?
@sureshanaparti as these are general bugfix/improvements for powerflex storage, should these be also applicable for 4.20 branch (esp these changes are scaleio/powerflex storage plugin related with no DB changes)?
@rohityadavcloud There are no DB changes, but changes in host SDC connection behavior earlier in this PR: https://github.com/apache/cloudstack/pull/9903 (already part of main), where the SDC connection is controlled using MDMs addition/removal instead of SDC service scini start/stop. This PR changes are on top of these, and introduces some configurations to validate the MDMs and apply timeout after add/remove MDMs. I think, it's better to keep this new behavior with main itself. A minor improvement (of keeping wait time after SDC service start/restart/stop, and retries to fetch SDC id/guid) is applicable for the old behavior as well, so can go in 4.20 branch. I've raised a separate PR for it here: https://github.com/apache/cloudstack/pull/11099
@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests
[SF] Trillian test result (tid-13615) Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8 Total time taken: 56231 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11047-t13615-kvm-ol8.zip Smoke tests completed. 139 look OK, 2 have errors, 0 did not run Only failed and skipped tests results shown below:
| Test | Result | Time (s) | Test File |
|---|---|---|---|
| test_02_enableHumanReadableLogs | Error |
0.23 | test_human_readable_logs.py |
| test_hostha_enable_ha_when_host_disabled | Error |
4.26 | test_hostha_kvm.py |
| test_hostha_enable_ha_when_host_in_maintenance | Error |
305.58 | test_hostha_kvm.py |
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Packaging result [SF]: ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 14008
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14013
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14072