cloudstack
cloudstack copied to clipboard
apache
New feature: Implicit host tags
Description
This PR adds a new feature to add implicit host tags via agent.properties
The goal of this PR is to provider CloudStack users another way to set host tags. For example, based on hardware which can be detected by scripts/commands, users can add different host tags (e.g. GPU, gpu type, SSD, raid type, network card type). Currently users have to use CloudStack API to manage the host tags. With this new feature, users can easily set the host tags in agent.properties by automation tools (chef, ansible, puppet, etc)
- (Explicit) host tags: the tags of host managed by cloudstack (CRUD APIs), including the flexible host tags
- Implicit host tags: the tags managed by agent.properties on kvm host
This PR also
- Merge two HostTagVO and HostTagDaoImpl
Doc PR: https://github.com/apache/cloudstack-documentation/pull/392
Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
- [ ] build/CI
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
- [ ] Major
- [ ] Minor
Bug Severity
- [ ] BLOCKER
- [ ] Critical
- [ ] Major
- [ ] Minor
- [ ] Trivial
Screenshots (if appropriate):
How Has This Been Tested?
How did you try to break this feature and the system with this change?
@weizhouapache, could you describe the objectives of this feature and how you expect it to work?
@weizhouapache, could you describe the objectives of this feature and how you expect it to work?
@GutoVeronezi I just added a paragraph in the description. Hope it answers your question.
@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Never mind about documentation. I couldn't read the linked doc PR above :man_facepalming:
Code looks good. Need to wait for the test results This should be useful for operators adding/re-adding specialized hosts as it makes adding host tags easier in such cases by only adding once.
thanks @shwstppr yeah it will be much easier for operators to set host tags.
Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9304
@weizhouapache a [SL] Trillian-Jenkins test job (rocky8 mgmt + kvm-rocky8) has been kicked to run smoke tests
[SF] Trillian test result (tid-9898) Environment: kvm-rocky8 (x2), Advanced Networking with Mgmt server r8 Total time taken: 48000 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8929-t9898-kvm-rocky8.zip Smoke tests completed. 128 look OK, 2 have errors, 0 did not run Only failed and skipped tests results shown below:
| Test | Result | Time (s) | Test File |
|---|---|---|---|
| test_01_events_resource | Error |
310.14 | test_events_resource.py |
| test_01_events_resource | Error |
310.15 | test_events_resource.py |
| test_04_deploy_vm_for_other_user_and_test_vm_operations | Failure |
96.39 | test_network_permissions.py |
| ContextSuite context=TestNetworkPermissions>:teardown | Error |
1.56 | test_network_permissions.py |
@JoaoJandre thanks for your review
the methods are moved from HostTagDaoImpl.java without any code changes. Since it has the very similar class name as HostTagsDaoImpl, I merged them into 1 file.
@JoaoJandre thanks for your review
the methods are moved from HostTagDaoImpl.java without any code changes. Since it has the very similar class name as HostTagsDaoImpl, I merged them into 1 file.
suggestion committed @JoaoJandre
actually the test could be simple, as the all host tags are considered during vm deployment and vm migration, etc, including the implicit and current (explicit) host tags.
@JoaoJandre thanks for your review the methods are moved from HostTagDaoImpl.java without any code changes. Since it has the very similar class name as HostTagsDaoImpl, I merged them into 1 file.
suggestion committed @JoaoJandre
actually the test could be simple, as the all host tags are considered during vm deployment and vm migration, etc, including the implicit and current (explicit) host tags.
@weizhouapache What happens if you define an implicit tag on a host and a flexible tag on it as well? Currently you either have normal or flexible tags, not both.
@JoaoJandre thanks for your review the methods are moved from HostTagDaoImpl.java without any code changes. Since it has the very similar class name as HostTagsDaoImpl, I merged them into 1 file.
suggestion committed @JoaoJandre
actually the test could be simple, as the all host tags are considered during vm deployment and vm migration, etc, including the implicit and current (explicit) host tags.
@weizhouapache What happens if you define an implicit tag on a host and a flexible tag on it as well? Currently you either have normal or flexible tags, not both.
here is a question: why not both? the normal tag could be regarded as a special type of flexible tag.
This PR will not break the flexible tags anyway, as the implicit tags are empty by default.
It can be regarded as a limitation. I've updated the doc PR. Anyone who is interested in supporting both implicit tags and flexible tags can work on the improvement to support both.
@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 4.26%. Comparing base (
f41c934) to head (3826fad). Report is 9 commits behind head on main.
Additional details and impacted files
@@ Coverage Diff @@
## main #8929 +/- ##
============================================
- Coverage 15.28% 4.26% -11.03%
============================================
Files 5425 364 -5061
Lines 473992 29614 -444378
Branches 59026 5197 -53829
============================================
- Hits 72439 1262 -71177
+ Misses 393510 28209 -365301
+ Partials 8043 143 -7900
| Flag | Coverage Δ | |
|---|---|---|
| uitests | 4.26% <ø> (-0.01%) |
:arrow_down: |
| unittests | ? |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9363
@weizhouapache a [SL] Trillian-Jenkins matrix job (centos7 mgmt + xenserver71, rocky8 mgmt + vmware67u3, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests
[SF] Trillian test result (tid-9959) Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7 Total time taken: 46026 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8929-t9959-kvm-centos7.zip Smoke tests completed. 128 look OK, 2 have errors, 0 did not run Only failed and skipped tests results shown below:
| Test | Result | Time (s) | Test File |
|---|---|---|---|
| test_01_events_resource | Error |
304.08 | test_events_resource.py |
| test_01_events_resource | Error |
304.09 | test_events_resource.py |
| test_04_deploy_vm_for_other_user_and_test_vm_operations | Failure |
88.48 | test_network_permissions.py |
| ContextSuite context=TestNetworkPermissions>:teardown | Error |
1.42 | test_network_permissions.py |
[SF] Trillian test result (tid-9958) Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server r8 Total time taken: 46346 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8929-t9958-vmware-67u3.zip Smoke tests completed. 119 look OK, 11 have errors, 0 did not run Only failed and skipped tests results shown below:
| Test | Result | Time (s) | Test File |
|---|---|---|---|
| test_01_events_resource | Error |
209.14 | test_events_resource.py |
| test_01_events_resource | Error |
209.15 | test_events_resource.py |
| test_DeployVmAntiAffinityGroup_in_project | Error |
122.00 | test_affinity_groups_projects.py |
| test_DeployVmAntiAffinityGroup | Error |
47.16 | test_affinity_groups.py |
| test_03_deploy_and_scale_kubernetes_cluster | Failure |
53.65 | test_kubernetes_clusters.py |
| test_08_upgrade_kubernetes_ha_cluster | Failure |
0.09 | test_kubernetes_clusters.py |
| test_04_deploy_vm_for_other_user_and_test_vm_operations | Failure |
104.01 | test_network_permissions.py |
| ContextSuite context=TestNetworkPermissions>:teardown | Error |
1.49 | test_network_permissions.py |
| test_01_non_strict_host_anti_affinity | Failure |
150.67 | test_nonstrict_affinity_group.py |
| test_02_non_strict_host_affinity | Error |
100.17 | test_nonstrict_affinity_group.py |
| test_01_verify_ipv6_vpc | Error |
3.45 | test_vpc_ipv6.py |
| test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL | Error |
5.79 | test_vpc_redundant.py |
| test_02_redundant_VPC_default_routes | Error |
5.86 | test_vpc_redundant.py |
| test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers | Error |
6.19 | test_vpc_redundant.py |
| test_04_rvpc_network_garbage_collector_nics | Error |
5.84 | test_vpc_redundant.py |
| test_05_rvpc_multi_tiers | Error |
5.59 | test_vpc_redundant.py |
| test_01_VPC_nics_after_destroy | Error |
6.41 | test_vpc_router_nics.py |
| test_02_VPC_default_routes | Error |
5.75 | test_vpc_router_nics.py |
| test_01_redundant_vpc_site2site_vpn | Failure |
3.79 | test_vpc_vpn.py |
| test_01_vpc_site2site_vpn_multiple_options | Failure |
3.49 | test_vpc_vpn.py |
| test_01_vpc_remote_access_vpn | Failure |
1.24 | test_vpc_vpn.py |
| test_01_vpc_site2site_vpn | Failure |
3.51 | test_vpc_vpn.py |
| test_01_cancel_host_maintenace_with_no_migration_jobs | Error |
0.08 | test_host_maintenance.py |
| test_02_cancel_host_maintenace_with_migration_jobs | Error |
0.07 | test_host_maintenance.py |
| test_03_cancel_host_maintenace_with_migration_jobs_failure | Error |
0.07 | test_host_maintenance.py |
[SF] Trillian test result (tid-9957) Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7 Total time taken: 48206 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8929-t9957-xenserver-71.zip Smoke tests completed. 128 look OK, 2 have errors, 0 did not run Only failed and skipped tests results shown below:
| Test | Result | Time (s) | Test File |
|---|---|---|---|
| test_01_events_resource | Error |
237.14 | test_events_resource.py |
| test_01_events_resource | Error |
237.14 | test_events_resource.py |
| test_04_deploy_vm_for_other_user_and_test_vm_operations | Failure |
107.84 | test_network_permissions.py |
| ContextSuite context=TestNetworkPermissions>:teardown | Error |
1.52 | test_network_permissions.py |
@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.
Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9401
@weizhouapache a [SL] Trillian-Jenkins test job (rocky8 mgmt + kvm-rocky8) has been kicked to run smoke tests