cloudstack icon indicating copy to clipboard operation
cloudstack copied to clipboard

Published 20 hours ago • verified publisher icon apache

Added support for storpool_qos service

Open slavkap opened this issue 11 months ago • 17 comments

Description

This PR provides support to use the storpool_qos service (to change tiers) via the Disk offering's details. StorPool provides the ‘storpool_qos’ service that tracks and configures the storage tier for all volumes based on a specifically provided qc tag specifying the storage tier for each volume.

Types of changes

  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [X] New feature (non-breaking change which adds functionality)
  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [ ] Enhancement (improves an existing feature and functionality)
  • [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
  • [ ] build/CI

Feature/Enhancement Scale

  • [ ] Major
  • [X] Minor

How Has This Been Tested?

manual and smoke tests

slavkap avatar Mar 07 '24 08:03 slavkap

Codecov Report

Attention: Patch coverage is 1.42180% with 208 lines in your changes are missing coverage. Please review.

Project coverage is 22.97%. Comparing base (9a73a2f) to head (93618c7). Report is 26 commits behind head on main.

Files Patch % Lines
...tastore/driver/StorPoolPrimaryDataStoreDriver.java 0.00% 138 Missing :warning:
...oudstack/storage/datastore/api/StorPoolVolume.java 0.00% 40 Missing :warning:
...loudstack/storage/datastore/util/StorPoolUtil.java 0.00% 17 Missing :warning:
...n/java/com/cloud/storage/VolumeApiServiceImpl.java 10.00% 9 Missing :warning:
.../subsystem/api/storage/PrimaryDataStoreDriver.java 0.00% 2 Missing :warning:
...udstack/storage/datastore/util/StorPoolHelper.java 0.00% 1 Missing :warning:
...in/java/com/cloud/storage/ResizeVolumePayload.java 66.66% 1 Missing :warning:
Additional details and impacted files 
@@             Coverage Diff              @@
##               main    #8755      +/-   ##
============================================
- Coverage     30.98%   22.97%   -8.01%     
+ Complexity    33526    23449   -10077     
============================================
  Files          5361     5262      -99     
  Lines        376250   357229   -19021     
  Branches      54933    51350    -3583     
============================================
- Hits         116564    82062   -34502     
- Misses       244259   263288   +19029     
+ Partials      15427    11879    -3548
Flag Coverage Δ
simulator-marvin-tests 24.61% <1.42%> (-0.21%) :arrow_down:
uitests 4.34% <ø> (-0.03%) :arrow_down:
unit-tests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Mar 07 '24 09:03 codecov[bot]

@blueorangutan package

sureshanaparti avatar Mar 11 '24 07:03 sureshanaparti

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Mar 11 '24 07:03 blueorangutan

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 8897

blueorangutan avatar Mar 11 '24 08:03 blueorangutan

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

github-actions[bot] avatar Mar 19 '24 12:03 github-actions[bot]

Codecov Report

Attention: Patch coverage is 0% with 264 lines in your changes missing coverage. Please review.

Project coverage is 15.57%. Comparing base (b61c3b8) to head (f956477). Report is 14 commits behind head on main.

Files with missing lines Patch % Lines
...tastore/driver/StorPoolPrimaryDataStoreDriver.java 0.00% 153 Missing :warning:
...stack/storage/datastore/api/StorPoolVolumeDef.java 0.00% 68 Missing :warning:
...loudstack/storage/datastore/util/StorPoolUtil.java 0.00% 21 Missing :warning:
...n/java/com/cloud/storage/VolumeApiServiceImpl.java 0.00% 10 Missing :warning:
...in/java/com/cloud/storage/ResizeVolumePayload.java 0.00% 6 Missing :warning:
.../subsystem/api/storage/PrimaryDataStoreDriver.java 0.00% 4 Missing :warning:
...udstack/storage/datastore/util/StorPoolHelper.java 0.00% 2 Missing :warning:
Additional details and impacted files 
@@             Coverage Diff              @@
##               main    #8755      +/-   ##
============================================
- Coverage     15.57%   15.57%   -0.01%     
- Complexity    12035    12041       +6     
============================================
  Files          5501     5505       +4     
  Lines        482219   482543     +324     
  Branches      58938    60302    +1364     
============================================
+ Hits          75119    75148      +29     
- Misses       398798   399091     +293     
- Partials       8302     8304       +2
Flag Coverage Δ
uitests 4.16% <ø> (-0.01%) :arrow_down:
unittests 16.35% <0.00%> (-0.01%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Apr 12 '24 07:04 codecov-commenter

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

github-actions[bot] avatar Apr 12 '24 13:04 github-actions[bot]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

github-actions[bot] avatar May 08 '24 15:05 github-actions[bot]

@blueorangutan package

DaanHoogland avatar May 18 '24 19:05 DaanHoogland

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar May 18 '24 19:05 blueorangutan

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 9630

blueorangutan avatar May 18 '24 20:05 blueorangutan

@blueorangutan test

DaanHoogland avatar May 22 '24 08:05 DaanHoogland

@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan avatar May 22 '24 08:05 blueorangutan

[SF] Trillian test result (tid-10243) Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7 Total time taken: 56225 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8755-t10243-kvm-centos7.zip Smoke tests completed. 126 look OK, 5 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_01_events_resource Error 425.39 test_events_resource.py
test_01_isolate_network_FW_PF_default_routes_egress_true Failure 112.26 test_routers_network_ops.py
test_03_secured_to_nonsecured_vm_migration Error 298.20 test_vm_life_cycle.py
test_01_verify_ipv6_vpc Failure 771.23 test_vpc_ipv6.py
test_02_redundant_VPC_default_routes Failure 385.82 test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers Failure 610.00 test_vpc_redundant.py
test_05_rvpc_multi_tiers Failure 530.74 test_vpc_redundant.py
test_05_rvpc_multi_tiers Error 530.76 test_vpc_redundant.py

blueorangutan avatar May 23 '24 00:05 blueorangutan

[SF] Trillian test result (tid-10264) Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7 Total time taken: 45150 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8755-t10264-kvm-centos7.zip Smoke tests completed. 130 look OK, 1 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_01_events_resource Error 412.21 test_events_resource.py

blueorangutan avatar May 23 '24 23:05 blueorangutan

@rp- @rg9975 @harikrishna-patnala can you review and maybe this? there is albeit minor changes in the generic code.

DaanHoogland avatar May 28 '24 05:05 DaanHoogland

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

github-actions[bot] avatar Jun 29 '24 02:06 github-actions[bot]

@blueorangutan package

DaanHoogland avatar Jul 03 '24 12:07 DaanHoogland

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Jul 03 '24 12:07 blueorangutan

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 10242

blueorangutan avatar Jul 03 '24 13:07 blueorangutan

@blueorangutan test

DaanHoogland avatar Jul 04 '24 07:07 DaanHoogland

@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan avatar Jul 04 '24 07:07 blueorangutan

[SF] Trillian test result (tid-10732) Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7 Total time taken: 57050 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8755-t10732-kvm-centos7.zip Smoke tests completed. 113 look OK, 24 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_role_account_acls_multiple_mgmt_servers Error 2.19 test_dynamicroles.py
test_query_async_job_result Error 102.90 test_async_job.py
test_revoke_certificate Error 0.01 test_certauthority_root.py
test_configure_ha_provider_invalid Error 0.01 test_hostha_simulator.py
test_configure_ha_provider_valid Error 0.01 test_hostha_simulator.py
test_ha_configure_enabledisable_across_clusterzones Error 0.01 test_hostha_simulator.py
test_ha_disable_feature_invalid Error 0.01 test_hostha_simulator.py
test_ha_enable_feature_invalid Error 0.01 test_hostha_simulator.py
test_ha_list_providers Error 0.01 test_hostha_simulator.py
test_ha_multiple_mgmt_server_ownership Error 0.01 test_hostha_simulator.py
test_ha_verify_fsm_available Error 0.01 test_hostha_simulator.py
test_ha_verify_fsm_degraded Error 0.01 test_hostha_simulator.py
test_ha_verify_fsm_fenced Error 0.01 test_hostha_simulator.py
test_ha_verify_fsm_recovering Error 0.01 test_hostha_simulator.py
test_hostha_configure_default_driver Error 0.01 test_hostha_simulator.py
test_hostha_configure_invalid_provider Error 0.01 test_hostha_simulator.py
test_hostha_disable_feature_valid Error 0.01 test_hostha_simulator.py
test_hostha_enable_feature_valid Error 0.01 test_hostha_simulator.py
test_hostha_enable_feature_without_setting_provider Error 0.01 test_hostha_simulator.py
test_list_ha_for_host Error 0.01 test_hostha_simulator.py
test_list_ha_for_host_invalid Error 0.01 test_hostha_simulator.py
test_list_ha_for_host_valid Error 0.01 test_hostha_simulator.py
test_01_host_ping_on_alert Error 0.07 test_host_ping.py
test_01_host_ping_on_alert Error 0.07 test_host_ping.py
test_01_browser_migrate_template Error 15.32 test_image_store_object_migration.py
test_01_invalid_upgrade_kubernetes_cluster Failure 239.32 test_kubernetes_clusters.py
test_02_upgrade_kubernetes_cluster Failure 236.78 test_kubernetes_clusters.py
test_03_deploy_and_scale_kubernetes_cluster Failure 226.04 test_kubernetes_clusters.py
test_04_autoscale_kubernetes_cluster Failure 239.71 test_kubernetes_clusters.py
test_05_basic_lifecycle_kubernetes_cluster Failure 233.54 test_kubernetes_clusters.py
test_06_delete_kubernetes_cluster Failure 213.79 test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster Failure 368.44 test_kubernetes_clusters.py
test_10_vpc_tier_kubernetes_cluster Failure 256.73 test_kubernetes_clusters.py
test_11_test_unmanaged_cluster_lifecycle Error 96.03 test_kubernetes_clusters.py
test_01_add_delete_kubernetes_supported_version Error 0.11 test_kubernetes_supported_versions.py
login_test_saml_user Error 2.89 test_login.py
test_01_deployVMInSharedNetwork Error 75.19 test_network.py
test_03_destroySharedNetwork Failure 1.08 test_network.py
ContextSuite context=TestSharedNetwork>:teardown Error 1.19 test_network.py
test_oobm_issue_power_cycle Error 3.29 test_outofbandmanagement_nestedplugin.py
test_oobm_issue_power_off Error 3.29 test_outofbandmanagement_nestedplugin.py
test_oobm_issue_power_on Error 3.28 test_outofbandmanagement_nestedplugin.py
test_oobm_issue_power_reset Error 3.30 test_outofbandmanagement_nestedplugin.py
test_oobm_issue_power_soft Error 3.31 test_outofbandmanagement_nestedplugin.py
test_oobm_issue_power_status Error 2.23 test_outofbandmanagement_nestedplugin.py
test_oobm_background_powerstate_sync Failure 21.61 test_outofbandmanagement.py
test_oobm_background_powerstate_sync Error 21.61 test_outofbandmanagement.py
test_oobm_configure_default_driver Error 0.05 test_outofbandmanagement.py
test_oobm_configure_invalid_driver Error 0.05 test_outofbandmanagement.py
test_oobm_disable_feature_invalid Error 0.04 test_outofbandmanagement.py
test_oobm_disable_feature_valid Error 1.14 test_outofbandmanagement.py
test_oobm_enable_feature_invalid Error 0.04 test_outofbandmanagement.py
test_oobm_enable_feature_valid Error 0.10 test_outofbandmanagement.py
test_oobm_enabledisable_across_clusterzones Error 10.82 test_outofbandmanagement.py
test_oobm_enabledisable_across_clusterzones Error 10.83 test_outofbandmanagement.py
test_oobm_issue_power_cycle Error 4.31 test_outofbandmanagement.py
test_oobm_issue_power_cycle Error 4.31 test_outofbandmanagement.py
test_oobm_issue_power_off Error 4.37 test_outofbandmanagement.py
test_oobm_issue_power_off Error 4.38 test_outofbandmanagement.py
test_oobm_issue_power_on Error 3.31 test_outofbandmanagement.py
test_oobm_issue_power_on Error 3.31 test_outofbandmanagement.py
test_oobm_issue_power_reset Error 4.33 test_outofbandmanagement.py
test_oobm_issue_power_reset Error 4.33 test_outofbandmanagement.py
test_oobm_issue_power_soft Error 4.32 test_outofbandmanagement.py
test_oobm_issue_power_soft Error 4.32 test_outofbandmanagement.py
test_oobm_issue_power_status Error 3.30 test_outofbandmanagement.py
test_oobm_issue_power_status Error 3.30 test_outofbandmanagement.py
test_oobm_multiple_mgmt_server_ownership Error 1.17 test_outofbandmanagement.py
test_oobm_multiple_mgmt_server_ownership Error 1.17 test_outofbandmanagement.py
test_oobm_zchange_password Error 1.19 test_outofbandmanagement.py
test_oobm_zchange_password Error 1.19 test_outofbandmanagement.py
test_02_edit_primary_storage_tags Error 0.02 test_primary_storage.py
test_01_primary_storage_scope_change Error 0.07 test_primary_storage_scope.py
test_01_vpc_privategw_acl Error 0.03 test_privategw_acl_ovs_gre.py
test_03_vpc_privategw_restart_vpc_cleanup Error 0.02 test_privategw_acl_ovs_gre.py
test_05_vpc_privategw_check_interface Error 0.01 test_privategw_acl_ovs_gre.py
test_01_vpc_privategw_acl Error 55.59 test_privategw_acl.py
test_02_vpc_privategw_static_routes Error 220.17 test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup Error 212.49 test_privategw_acl.py
test_04_rvpc_privategw_static_routes Error 353.19 test_privategw_acl.py
test_01_purge_expunged_api_vm_start_date Error 61.00 test_purge_expunged_vms.py
test_02_purge_expunged_api_vm_end_date Error 49.54 test_purge_expunged_vms.py
test_03_purge_expunged_api_vm_start_end_date Error 46.05 test_purge_expunged_vms.py
test_04_purge_expunged_api_vm_no_date Error 46.00 test_purge_expunged_vms.py
test_05_purge_expunged_vm_service_offering Error 274.19 test_purge_expunged_vms.py
test_06_purge_expunged_vm_background_task Error 345.19 test_purge_expunged_vms.py
test_01_snapshot_root_disk Error 3.30 test_snapshots.py
test_CreateTemplateWithDuplicateName Error 23.84 test_templates.py
test_01_register_template_direct_download_flag Error 0.16 test_templates.py
test_01_positive_tests_usage Error 9.45 test_usage_events.py
test_01_ISO_usage Error 1.10 test_usage.py
test_01_lb_usage Error 4.28 test_usage.py
test_01_nat_usage Error 8.30 test_usage.py
test_01_public_ip_usage Error 1.07 test_usage.py
test_01_snapshot_usage Error 19.57 test_usage.py
test_01_template_usage Error 13.50 test_usage.py
test_01_vm_usage Error 132.92 test_usage.py
test_01_volume_usage Error 127.98 test_usage.py
test_01_vpn_usage Error 9.47 test_usage.py
test_02_unsecure_vm_migration Error 533.05 test_vm_life_cycle.py
test_02_unsecure_vm_migration Error 533.06 test_vm_life_cycle.py
test_03_secured_to_nonsecured_vm_migration Error 0.02 test_vm_life_cycle.py
test_04_nonsecured_to_secured_vm_migration Error 0.01 test_vm_life_cycle.py
test_12_start_vm_multiple_volumes_allocated Error 10.57 test_vm_life_cycle.py
test_01_vmschedule_create Error 0.10 test_vm_schedule.py
test_disable_oobm_ha_state_ineligible Error 0.05 test_hostha_kvm.py
test_hostha_configure_default_driver Error 0.04 test_hostha_kvm.py
test_hostha_enable_ha_when_host_disabled Error 0.04 test_hostha_kvm.py
test_hostha_enable_ha_when_host_disconected Error 0.04 test_hostha_kvm.py
test_hostha_enable_ha_when_host_in_maintenance Error 0.04 test_hostha_kvm.py
test_hostha_kvm_host_degraded Error 0.04 test_hostha_kvm.py
test_hostha_kvm_host_fencing Error 0.04 test_hostha_kvm.py
test_hostha_kvm_host_recovering Error 0.04 test_hostha_kvm.py
test_remove_ha_provider_not_possible Error 0.04 test_hostha_kvm.py

blueorangutan avatar Jul 05 '24 00:07 blueorangutan

not sure if this requires more testing than smoke tests (for other storage solutions cc @harikrishna-patnala @rp- @rg9975 ) ???

DaanHoogland avatar Aug 28 '24 09:08 DaanHoogland

@blueorangutan package

DaanHoogland avatar Aug 28 '24 09:08 DaanHoogland

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Aug 28 '24 09:08 blueorangutan

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10841

blueorangutan avatar Aug 28 '24 11:08 blueorangutan

@blueorangutan test

DaanHoogland avatar Aug 28 '24 11:08 DaanHoogland

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan avatar Aug 28 '24 11:08 blueorangutan

not sure if this requires more testing than smoke tests (for other storage solutions cc @harikrishna-patnala @rp- @rg9975 ) ???

I don't think this can in anyway break the Linstor driver

rp- avatar Aug 28 '24 11:08 rp-