HAproxy based load balancer Customisation

[CharlesQueiroz]

Description

Add a new feature for load balancers based on haproxy.

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/VR+haproxy+customization+in+CloudStack

Fixes https://github.com/apache/cloudstack/issues/3789

This commit was based on this:

https://github.com/apache/cloudstack/pull/4141 and https://github.com/apache/cloudstack/pull/5799

It supports (or will support)

Basic proxy configurations: haproxy statistics uri, auth, enable/disable global maxconn and maxpipes Basic configurations on rule timeout connection, client ,server globally and per rule http, httpalive per rule maxconn, fullconn per rule maxconn, minconn, maxqueue per server in rule advanced features transparent load balancer SSL offloading http2 support Variable SSL configurations

Types of changes

• [ ] Breaking change (fix or feature that would cause existing functionality to change)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Bug fix (non-breaking change which fixes an issue)
• [x] Enhancement (improves an existing feature and functionality)
• [ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• [x] Major
• [ ] Minor

Screenshots (if appropriate):

Adding networks level haproxy configs

Adding load balancer level configs

Adding SSL Certificate

[codecov[bot]]

Codecov Report

:x: Patch coverage is 0.15432% with 647 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 13.81%. Comparing base (b205548) to head (c18f05d). :warning: Report is 1866 commits behind head on main.

Files with missing lines	Patch %	Lines
...tack/network/lb/LoadBalancerConfigManagerImpl.java	0.00%	187 Missing :warning:
ui/src/views/network/LoadBalancing.vue	0.00%	102 Missing :warning:
...ork/router/VirtualNetworkApplianceManagerImpl.java	0.00%	92 Missing :warning:
...m/cloud/network/dao/LoadBalancerConfigDaoImpl.java	0.00%	79 Missing :warning:
ui/src/views/network/LbConfigTab.vue	0.00%	57 Missing :warning:
...src/main/java/com/cloud/api/ApiResponseHelper.java	0.00%	48 Missing :warning:
...va/com/cloud/network/dao/LoadBalancerConfigVO.java	0.00%	38 Missing :warning:
...a/com/cloud/network/router/CommandSetupHelper.java	0.00%	17 Missing :warning:
...loud/network/lb/LoadBalancingRulesManagerImpl.java	0.00%	8 Missing and 1 partial :warning:
.../cloud/configuration/ConfigurationManagerImpl.java	0.00%	8 Missing :warning:
... and 3 more

:exclamation: There is a different number of reports uploaded between BASE (b205548) and HEAD (c18f05d). Click for more details.

HEAD has 8 uploads less than BASE

Flag	BASE (b205548)	HEAD (c18f05d)
simulator-marvin-tests	8	0

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #7484      +/-   ##
============================================
- Coverage     19.87%   13.81%   -6.06%     
+ Complexity    19052    10440    -8612     
============================================
  Files          4966     3094    -1872     
  Lines        336442   291065   -45377     
  Branches      48347    47361     -986     
============================================
- Hits          66869    40221   -26648     
+ Misses       259830   245653   -14177     
+ Partials       9743     5191    -4552     

Flag	Coverage Δ
simulator-marvin-tests	?
uitests	4.84% <0.00%> (-0.04%)	:arrow_down:
unit-tests	14.69% <0.20%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[CharlesQueiroz]

Hi @ustcweizhou,

I recently opened this pull request with HAProxy config changes. It would be great to have your opinion on my changes.

I have reviewed the code and run some tests, but I'd like your opinion to ensure that the code follows best practices and is scalable for future implementations. I'd appreciate you looking at my pull request and giving me some suggestions or pointing out any problems you find. It would be essential to get your opinion and expertise.

Thank you very much in advance for your help and time.

[weizhouapache]

Hi @ustcweizhou,

I recently opened this pull request with HAProxy config changes. It would be great to have your opinion on my changes.

I have reviewed the code and run some tests, but I'd like your opinion to ensure that the code follows best practices and is scalable for future implementations. I'd appreciate you looking at my pull request and giving me some suggestions or pointing out any problems you find. It would be essential to get your opinion and expertise.

Thank you very much in advance for your help and time.

Hi @CharlesQueiroz good to hear from you. I am currently busy on some other stuffs, I will have a review.

cc @rohityadavcloud @DaanHoogland @shwstppr what do you think of this in 4.19 ?

[rohityadavcloud]

@blueorangutan package

[blueorangutan]

@rohityadavcloud a [SF] Jenkins job has been kicked to build packages. It will be bundled with SystemVM template(s). I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: el9 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 6154

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[DaanHoogland]

@blueorangutan package

[blueorangutan]

@DaanHoogland a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[DaanHoogland]

@CharlesQueiroz can you look into the failing GitHub actions, please?

[CharlesQueiroz]

Sure. I'm looking...

[blueorangutan]

Packaging result [SF]: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: el9 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 7043

[DaanHoogland]

@blueorangutan package

[blueorangutan]

@DaanHoogland a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: el9 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 7049

[DaanHoogland]

@blueorangutan package

[blueorangutan]

@DaanHoogland a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: el9 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 7060

[DaanHoogland]

@blueorangutan test

[blueorangutan]

@DaanHoogland a [SF] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian test result (tid-7692) Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7 Total time taken: 28572 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7484-t7692-kvm-centos7.zip Smoke tests completed. 52 look OK, 61 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_deploy_vm_from_iso	Error	21.54	test_deploy_vm_iso.py
ContextSuite context=TestAddVmToSubDomain>:setup	Error	14.86	test_accounts.py
test_DeleteDomain	Error	21.04	test_accounts.py
test_forceDeleteDomain	Failure	20.85	test_accounts.py
test_01_user_remove_VM_running	Error	6.24	test_accounts.py
test_01_non_strict_host_anti_affinity	Error	6.52	test_nonstrict_affinity_group.py
test_02_non_strict_host_affinity	Error	7.51	test_nonstrict_affinity_group.py
test_delete_account	Error	21.62	test_network.py
ContextSuite context=TestPortForwarding>:setup	Error	51.57	test_network.py
ContextSuite context=TestPublicIP>:setup	Error	9.45	test_network.py
test_reboot_router	Error	6.44	test_network.py
test_releaseIP	Error	7.36	test_network.py
ContextSuite context=TestRouterRules>:setup	Error	13.86	test_network.py
test_01_deployVMInSharedNetwork	Failure	5.28	test_network.py
test_02_verifyRouterIpAfterNetworkRestart	Failure	4.14	test_network.py
test_03_destroySharedNetwork	Failure	1.08	test_network.py
ContextSuite context=TestSharedNetwork>:teardown	Error	2.17	test_network.py
test_DeployVmAffinityGroup	Error	5.38	test_affinity_groups.py
test_DeployVmAntiAffinityGroup	Error	4.32	test_affinity_groups.py
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80	Error	45.98	test_internal_lb.py
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80	Error	45.99	test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80	Error	94.41	test_internal_lb.py
test_02_internallb_roundrobin_1RVPC_3VM_HTTP_port80	Error	94.42	test_internal_lb.py
test_03_vpc_internallb_haproxy_stats_on_all_interfaces	Error	43.92	test_internal_lb.py
test_03_vpc_internallb_haproxy_stats_on_all_interfaces	Error	43.93	test_internal_lb.py
test_04_rvpc_internallb_haproxy_stats_on_all_interfaces	Error	90.66	test_internal_lb.py
test_04_rvpc_internallb_haproxy_stats_on_all_interfaces	Error	90.67	test_internal_lb.py
test_network_acl	Error	53.06	test_network_acl.py
ContextSuite context=TestLBRuleUsage>:setup	Error	9.48	test_usage.py
ContextSuite context=TestNatRuleUsage>:setup	Error	15.96	test_usage.py
ContextSuite context=TestPublicIPUsage>:setup	Error	22.23	test_usage.py
ContextSuite context=TestSnapshotUsage>:setup	Error	29.51	test_usage.py
ContextSuite context=TestVmUsage>:setup	Error	50.30	test_usage.py
ContextSuite context=TestVolumeUsage>:setup	Error	57.61	test_usage.py
ContextSuite context=TestVpnUsage>:setup	Error	63.86	test_usage.py
test_DeployVmAntiAffinityGroup_in_project	Error	5.37	test_affinity_groups_projects.py
test_deploy_vm_with_registered_userdata	Error	8.58	test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_allow	Error	7.61	test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_append	Error	8.56	test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_params	Error	7.61	test_register_userdata.py
ContextSuite context=TestAnnotations>:setup	Error	0.00	test_annotations.py
ContextSuite context=TestRemoteDiagnostics>:setup	Error	0.00	test_diagnostics.py
ContextSuite context=TestResetVmOnReboot>:setup	Error	0.00	test_reset_vm_on_reboot.py
test_query_async_job_result	Error	15.49	test_async_job.py
ContextSuite context=TestIsolatedNetworksPasswdServer>:setup	Error	0.00	test_password_server.py
test_nic_secondaryip_add_remove	Error	22.47	test_multipleips_per_nic.py
ContextSuite context=TestMultipleVolumeAttach>:setup	Error	0.00	test_attach_multiple_volumes.py
test_01_add_primary_storage_disabled_host	Error	0.17	test_primary_storage.py
test_01_primary_storage_nfs	Error	0.15	test_primary_storage.py
ContextSuite context=TestStorageTags>:setup	Error	0.25	test_primary_storage.py
test_01_invalid_upgrade_kubernetes_cluster	Failure	7.31	test_kubernetes_clusters.py
test_02_upgrade_kubernetes_cluster	Failure	8.33	test_kubernetes_clusters.py
test_03_deploy_and_scale_kubernetes_cluster	Failure	8.31	test_kubernetes_clusters.py
test_04_autoscale_kubernetes_cluster	Failure	7.29	test_kubernetes_clusters.py
test_05_basic_lifecycle_kubernetes_cluster	Failure	6.25	test_kubernetes_clusters.py
test_06_delete_kubernetes_cluster	Failure	7.31	test_kubernetes_clusters.py
test_07_deploy_kubernetes_ha_cluster	Failure	6.50	test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster	Failure	6.26	test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster	Failure	8.32	test_kubernetes_clusters.py
test_10_vpc_tier_kubernetes_cluster	Failure	50.98	test_kubernetes_clusters.py
test_11_test_unmanaged_cluster_lifecycle	Error	1.23	test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown	Error	77.02	test_kubernetes_clusters.py
test_01_verify_ipv6_network	Error	5.88	test_network_ipv6.py
test_01_verify_ipv6_network	Error	5.88	test_network_ipv6.py
test_01_nic	Error	57.30	test_nic.py
ContextSuite context=TestConsoleEndpoint>:setup	Error	0.00	test_console_endpoint.py
ContextSuite context=TestSnapshotRootDisk>:setup	Error	0.00	test_snapshots.py
test_01_native_to_native_network_migration	Error	17.84	test_migration.py
test_02_native_to_native_vpc_migration	Error	44.30	test_migration.py
ContextSuite context=TestDeployVirtioSCSIVM>:setup	Error	0.00	test_deploy_virtio_scsi_vm.py
test_02_deploy_vm_with_extraconfig_kvm	Error	4.35	test_deploy_vm_extra_config_data.py
test_03_update_vm_with_extraconfig_kvm	Error	4.32	test_deploy_vm_extra_config_data.py
test_01_isolated_persistent_network	Error	2.91	test_persistent_network.py
ContextSuite context=TestL2PersistentNetworks>:teardown	Error	48.64	test_persistent_network.py
test_00_deploy_vm_root_resize	Error	5.42	test_deploy_vm_root_resize.py
test_deployvm_userdata	Error	4.55	test_deploy_vm_with_userdata.py
test_deployvm_userdata_post	Error	4.44	test_deploy_vm_with_userdata.py
test_deploy_more_vms_than_limit_allows	Error	6.40	test_deploy_vms_in_parallel.py
test_deployvm_firstfit	Error	5.41	test_deploy_vms_with_varied_deploymentplanners.py
test_deployvm_userconcentrated	Error	4.33	test_deploy_vms_with_varied_deploymentplanners.py
test_deployvm_userdispersing	Error	3.34	test_deploy_vms_with_varied_deploymentplanners.py
test_01_so_removal_resource_update	Error	5.52	test_resource_accounting.py
test_01_create_delete_portforwarding_fornonvpc	Error	7.59	test_portforwardingrules.py
test_03_network_operations_on_created_vm_of_otheruser	Error	6.50	test_network_permissions.py
test_03_network_operations_on_created_vm_of_otheruser	Error	6.50	test_network_permissions.py
test_04_deploy_vm_for_other_user_and_test_vm_operations	Failure	4.38	test_network_permissions.py
ContextSuite context=TestNetworkPermissions>:teardown	Error	1.34	test_network_permissions.py
test_03_deploy_vm_domain_service_offering	Error	19.62	test_domain_service_offerings.py
test_02_vpc_privategw_static_routes	Failure	82.24	test_privategw_acl.py
test_02_vpc_privategw_static_routes	Error	82.25	test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup	Failure	87.03	test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup	Error	87.04	test_privategw_acl.py
test_04_rvpc_privategw_static_routes	Failure	181.74	test_privategw_acl.py
test_04_rvpc_privategw_static_routes	Error	181.75	test_privategw_acl.py
test_01_events_resource	Error	7.57	test_events_resource.py
test_01_events_resource	Error	7.57	test_events_resource.py
test_updating_nics_on_two_shared_networks	Error	4.59	test_gateway_on_shared_networks.py
ContextSuite context=TestGatewayOnSharedNetwork>:teardown	Error	6.74	test_gateway_on_shared_networks.py
ContextSuite context=TestRouterDHCPHosts>:setup	Error	0.00	test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPOpts>:setup	Error	0.00	test_router_dhcphosts.py
test_09_project_suspend	Error	5.47	test_projects.py
test_10_project_activation	Error	6.38	test_projects.py
ContextSuite context=TestHostControlState>:setup	Error	34.48	test_host_control_state.py
ContextSuite context=TestListIdsParams>:setup	Error	0.00	test_list_ids_parameter.py
ContextSuite context=TestRouterServices>:setup	Error	0.00	test_routers.py
ContextSuite context=TestRouterDns>:setup	Error	0.00	test_router_dns.py
ContextSuite context=TestLoadBalance>:setup	Error	0.00	test_loadbalance.py
ContextSuite context=TestRouterDnsService>:setup	Error	0.00	test_router_dnsservice.py
test_list_vms_metrics_admin	Error	5.81	test_metrics_api.py
test_list_vms_metrics_history	Error	5.62	test_metrics_api.py
test_list_vms_metrics_user	Error	5.61	test_metrics_api.py
test_list_volumes_metrics_history	Error	4.46	test_metrics_api.py
test_01_isolate_network_FW_PF_default_routes_egress_true	Error	4.53	test_routers_network_ops.py
test_02_isolate_network_FW_PF_default_routes_egress_false	Error	5.56	test_routers_network_ops.py
test_02_routervm_iptables_policies	Error	4.38	test_routers_iptables_default_policy.py
test_01_single_VPC_iptables_policies	Error	60.16	test_routers_iptables_default_policy.py
test_01_single_VPC_iptables_policies	Error	60.16	test_routers_iptables_default_policy.py
ContextSuite context=TestCpuCapServiceOfferings>:setup	Error	0.00	test_service_offerings.py
ContextSuite context=TestServiceOfferings>:setup	Error	7.47	test_service_offerings.py
test_02_deploy_vm_from_direct_download_template	Error	5.41	test_templates.py
ContextSuite context=TestTemplates>:setup	Error	37.47	test_templates.py
test_01_scale_up_verify	Failure	576.43	test_vm_autoscaling.py
test_02_update_vmprofile_and_vmgroup	Failure	245.60	test_vm_autoscaling.py
test_03_scale_down_verify	Failure	304.55	test_vm_autoscaling.py
test_04_stop_remove_vm_in_vmgroup	Failure	14.70	test_vm_autoscaling.py
test_06_autoscaling_vmgroup_on_project_network	Failure	587.55	test_vm_autoscaling.py
test_06_autoscaling_vmgroup_on_project_network	Error	587.56	test_vm_autoscaling.py
test_07_autoscaling_vmgroup_on_vpc_network	Failure	614.32	test_vm_autoscaling.py
test_07_autoscaling_vmgroup_on_vpc_network	Error	614.33	test_vm_autoscaling.py
ContextSuite context=TestVmAutoScaling>:teardown	Error	628.17	test_vm_autoscaling.py
test_01_deploy_vm_on_specific_host	Error	0.07	test_vm_deployment_planner.py
test_02_deploy_vm_on_specific_cluster	Error	4.36	test_vm_deployment_planner.py
test_03_deploy_vm_on_specific_pod	Error	2.26	test_vm_deployment_planner.py
test_04_deploy_vm_on_host_override_pod_and_cluster	Error	0.10	test_vm_deployment_planner.py
test_05_deploy_vm_on_cluster_override_pod	Error	2.25	test_vm_deployment_planner.py
ContextSuite context=TestDeployVM>:setup	Error	0.00	test_vm_life_cycle.py
test_01_secure_vm_migration	Error	64.97	test_vm_life_cycle.py
test_02_unsecure_vm_migration	Error	213.49	test_vm_life_cycle.py
test_03_secured_to_nonsecured_vm_migration	Error	139.97	test_vm_life_cycle.py
test_04_nonsecured_to_secured_vm_migration	Error	141.73	test_vm_life_cycle.py
ContextSuite context=TestVMLifeCycle>:setup	Error	6.59	test_vm_life_cycle.py
ContextSuite context=TestVMSchedule>:setup	Error	0.00	test_vm_schedule.py
ContextSuite context=TestVmSnapshot>:setup	Error	6.46	test_vm_snapshots.py
ContextSuite context=TestCreateVolume>:setup	Error	0.00	test_volumes.py
ContextSuite context=TestVolumes>:setup	Error	7.69	test_volumes.py
test_01_verify_ipv6_vpc	Error	50.20	test_vpc_ipv6.py
test_01_verify_ipv6_vpc	Error	50.21	test_vpc_ipv6.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL	Failure	113.33	test_vpc_redundant.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL	Error	113.33	test_vpc_redundant.py
test_02_redundant_VPC_default_routes	Failure	108.12	test_vpc_redundant.py
test_02_redundant_VPC_default_routes	Error	108.13	test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers	Failure	114.33	test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers	Error	114.34	test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics	Failure	112.22	test_vpc_redundant.py
test_04_rvpc_network_garbage_collector_nics	Error	112.23	test_vpc_redundant.py
test_05_rvpc_multi_tiers	Failure	110.23	test_vpc_redundant.py
test_05_rvpc_multi_tiers	Error	110.24	test_vpc_redundant.py
test_01_VPC_nics_after_destroy	Failure	55.18	test_vpc_router_nics.py
test_02_VPC_default_routes	Failure	55.93	test_vpc_router_nics.py
test_01_redundant_vpc_site2site_vpn	Failure	178.63	test_vpc_vpn.py
test_01_redundant_vpc_site2site_vpn	Error	178.64	test_vpc_vpn.py
test_01_vpc_site2site_vpn_multiple_options	Failure	83.70	test_vpc_vpn.py
test_01_vpc_site2site_vpn_multiple_options	Error	83.70	test_vpc_vpn.py
test_01_vpc_remote_access_vpn	Error	46.46	test_vpc_vpn.py
test_01_vpc_remote_access_vpn	Error	46.47	test_vpc_vpn.py
test_01_vpc_site2site_vpn	Failure	80.58	test_vpc_vpn.py
test_01_vpc_site2site_vpn	Error	80.59	test_vpc_vpn.py

[DaanHoogland]

@blueorangutan package

[blueorangutan]

@DaanHoogland a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: el9 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 7150

[DaanHoogland]

@blueorangutan test

[DaanHoogland]

@CharlesQueiroz can you look at the github action failures?

[DaanHoogland]

@CharlesQueiroz (cc @weizhouapache ) will this be done in time for 4.19 or should we postpone?

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.