Improve logs when adding components to avoid set

[SadiJr]

Description

The process executed by ACS when deploying a new VM is complex, and without enough logs to identify why one component is added to avoid set. Thus, new logs were added in the process of adding a component to avoid set, and also some methods were split to sub methods, aiming to facilitate future troubleshootings, and increase legibility.

Types of changes

• [ ] Breaking change (fix or feature that would cause existing functionality to change)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Bug fix (non-breaking change which fixes an issue)
• [ ] Enhancement (improves an existing feature and functionality)
• [x] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Bug Severity

• [ ] BLOCKER
• [ ] Critical
• [ ] Major
• [x] Minor
• [ ] Trivial

Screenshots (if appropriate):

How Has This Been Tested?

It was tested in a local lab:

1. I deployed new VMs;
2. I read the logs to see if they were improved;
3. Before, the logs did not have enough information to perform an efficient troubleshooting;
4. Now, with improved logs, the troubleshooting will be easier.

[sonarqubecloud[bot]]

SonarCloud Quality Gate failed.   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
7 Code Smells

14.4% Coverage
0.0% Duplication

[kiranchavala]

@blueorangutan package

[blueorangutan]

@kiranchavala a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: el9 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 5689

[kiranchavala]

@SadiJr

Could you please provide a sample snippet of the new improved logs so that i can verify it.

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[SadiJr]

@SadiJr

Could you please provide a sample snippet of the new improved logs so that i can verify it.

@kiranchavala sorry for the delay to answer you, sure I can provide some samples:

2023-08-10 11:22:47,703 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-96:ctx-1407e23c job-18142/job-18143 ctx-5428ad01) (logid:1473ee27) Trying to start VM ["a52c2c99-3754-4b64-bfc9-389baac852e1"] using plan [{"_dcId":1,"_podId":1,"_clusterId":6,"_hostId":3223935,"_recreateDisks":false,"preferredHostIds":[],"migrationPlan":false,"hostPriorities":{}}] and planner [].
...
2023-08-10 11:22:47,714 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-96:ctx-1407e23c job-18142/job-18143 ctx-5428ad01) (logid:1473ee27) Trying to deploy VM [error decoding VM instance {"id":1308,"instanceName":"i-134-1308-VM","type":"User","uuid":"a52c2c99-3754-4b64-bfc9-389baac852e1"}] and details: Plan [{"_dcId":1,"_podId":1,"_clusterId":6,"_hostId":3223935,"_recreateDisks":false,"preferredHostIds":[],"migrationPlan":false,"hostPriorities":{}}]; avoid list [] and planner: [].
2023-08-10 11:22:47,715 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-96:ctx-1407e23c job-18142/job-18143 ctx-5428ad01) (logid:1473ee27) Avoiding components [] in deployment of VM ["a52c2c99-3754-4b64-bfc9-389baac852e1"].

Please tell me if you need more information.

[DaanHoogland]

@kiranchavala are you still looking to test this?

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 55.00000% with 126 lines in your changes are missing coverage. Please review.

Project coverage is 30.93%. Comparing base (a31449b) to head (2d4dc6b). Report is 7 commits behind head on main.

Files	Patch %	Lines
...om/cloud/deploy/DeploymentPlanningManagerImpl.java	56.80%	70 Missing and 41 partials :warning:
.../main/java/com/cloud/deploy/DeploymentPlanner.java	8.33%	11 Missing :warning:
...gent/manager/allocator/impl/FirstFitAllocator.java	33.33%	4 Missing :warning:

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #7214      +/-   ##
============================================
+ Coverage     30.76%   30.93%   +0.16%     
- Complexity    33076    33425     +349     
============================================
  Files          5353     5355       +2     
  Lines        374607   375777    +1170     
  Branches      54635    54897     +262     
============================================
+ Hits         115238   116234     +996     
- Misses       244099   244139      +40     
- Partials      15270    15404     +134     

Flag	Coverage Δ
simulator-marvin-tests	24.78% <53.21%> (+0.15%)	:arrow_up:
uitests	4.36% <ø> (-0.02%)	:arrow_down:
unit-tests	16.56% <22.50%> (+0.12%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[kiranchavala]

@blueorangutan package

[blueorangutan]

@kiranchavala a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: el9 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 7142

[shwstppr]

@blueorangutan package

[blueorangutan]

@shwstppr a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: el9 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 7208

[DaanHoogland]

@shwstppr are we going to merge this?

[shwstppr]

@shwstppr are we going to merge this?

Lets run a round of test matrix

@blueorangutan package

[blueorangutan]

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: el9 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 7781

[DaanHoogland]

@blueorangutan test matrix

[blueorangutan]

@DaanHoogland a [SL] Trillian-Jenkins matrix job (centos7 mgmt + xenserver71, rocky8 mgmt + vmware67u3, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian Build Failed (tid-8346)

[blueorangutan]

[SF] Trillian Build Failed (tid-8347)

[blueorangutan]

[SF] Trillian Build Failed (tid-8348)

[blueorangutan]

[SF] Trillian test result (tid-8358) Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7 Total time taken: 51845 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7214-t8358-kvm-centos7.zip Smoke tests completed. 115 look OK, 3 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_08_upgrade_kubernetes_ha_cluster	Failure	681.93	test_kubernetes_clusters.py
test_08_migrate_vm	Error	45.85	test_vm_life_cycle.py
test_05_rvpc_multi_tiers	Failure	444.21	test_vpc_redundant.py
test_05_rvpc_multi_tiers	Error	444.23	test_vpc_redundant.py

[blueorangutan]

[SF] Trillian test result (tid-8357) Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server r8 Total time taken: 56569 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7214-t8357-vmware-67u3.zip Smoke tests completed. 113 look OK, 5 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_browser_migrate_template	Failure	5.42	test_image_store_object_migration.py
test_01_add_primary_storage_disabled_host	Error	36.01	test_primary_storage.py
test_01_sys_vm_start	Failure	0.09	test_secondary_storage.py
test_01_deploy_vm_on_specific_host	Error	0.08	test_vm_deployment_planner.py
test_02_deploy_vm_on_specific_cluster	Error	3602.49	test_vm_deployment_planner.py
test_03_deploy_vm_on_specific_pod	Error	1.34	test_vm_deployment_planner.py
test_04_deploy_vm_on_host_override_pod_and_cluster	Error	0.12	test_vm_deployment_planner.py
test_05_deploy_vm_on_cluster_override_pod	Error	4.43	test_vm_deployment_planner.py
test_03_live_migrate_VM_with_two_data_disks	Error	61.94	test_vm_life_cycle.py
test_08_migrate_vm	Error	0.06	test_vm_life_cycle.py

[blueorangutan]

[SF] Trillian test result (tid-8369) Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7 Total time taken: 45929 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7214-t8369-xenserver-71.zip Smoke tests completed. 113 look OK, 5 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestSharedNetwork>:setup	Error	56.25	test_network.py
test_01_add_primary_storage_disabled_host	Error	42.55	test_primary_storage.py
test_01_sys_vm_start	Failure	0.06	test_secondary_storage.py
test_01_deploy_vm_on_specific_host	Error	0.09	test_vm_deployment_planner.py
test_04_deploy_vm_on_host_override_pod_and_cluster	Error	0.12	test_vm_deployment_planner.py
test_08_migrate_vm	Error	0.06	test_vm_life_cycle.py

[shwstppr]

Some failures in deployment planner tests, retry?

@blueorangutan package

[blueorangutan]

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: el9 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 7813