Enable Security groups for Shared networks in Advanced zone without security groups

[vishesh92]

Description

This PR allows creating shared networks with Security Groups in a Zone with securitygroupenabled set to false.

To enable SG, enable the SecurityGroupProvider for Guest physical network. After this change, you will be able to create a shared network with Security Groups.

Types of changes

• [ ] Breaking change (fix or feature that would cause existing functionality to change)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Bug fix (non-breaking change which fixes an issue)
• [x] Enhancement (improves an existing feature and functionality)
• [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
• [ ] build/CI
• [ ] test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• [ ] Major
• [x] Minor

Bug Severity

• [ ] BLOCKER
• [ ] Critical
• [ ] Major
• [ ] Minor
• [ ] Trivial

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

[vishesh92]

@blueorangutan package

[blueorangutan]

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 12.24490% with 43 lines in your changes missing coverage. Please review.

Project coverage is 15.57%. Comparing base (47a6b70) to head (5637b12). Report is 92 commits behind head on main.

Files with missing lines	Patch %	Lines
.../main/java/com/cloud/network/NetworkModelImpl.java	12.00%	19 Missing and 3 partials :warning:
...g/apache/cloudstack/api/response/ZoneResponse.java	0.00%	9 Missing :warning:
...er/actionworkers/KubernetesClusterStartWorker.java	0.00%	5 Missing :warning:
...isor/xenserver/discoverer/XcpServerDiscoverer.java	0.00%	2 Missing :warning:
...loudstack/api/command/user/vm/AddIpToVmNicCmd.java	0.00%	1 Missing :warning:
...tack/api/command/user/vm/RemoveIpFromVmNicCmd.java	0.00%	1 Missing :warning:
...KubernetesClusterResourceModifierActionWorker.java	0.00%	1 Missing :warning:
.../src/main/java/com/cloud/vm/UserVmManagerImpl.java	50.00%	1 Missing :warning:
...stack/storage/template/VnfTemplateManagerImpl.java	0.00%	0 Missing and 1 partial :warning:

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #9415      +/-   ##
============================================
+ Coverage     15.53%   15.57%   +0.03%     
- Complexity    11988    12049      +61     
============================================
  Files          5496     5505       +9     
  Lines        481450   482703    +1253     
  Branches      60738    62405    +1667     
============================================
+ Hits          74809    75187     +378     
- Misses       398364   399209     +845     
- Partials       8277     8307      +30     

Flag	Coverage Δ
uitests	4.16% <ø> (-0.02%)	:arrow_down:
unittests	16.35% <12.24%> (+0.04%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[blueorangutan]

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10398

[weizhouapache]

@blueorangutan test alma9 kvm-alma9

[blueorangutan]

@weizhouapache a [SL] Trillian-Jenkins test job (alma9 mgmt + kvm-alma9) has been kicked to run smoke tests

[vishesh92]

@weizhouapache This PR is still in progress and needs more work. All the simulators tests have also failed. Add WIP in the PR title for now.

[vishesh92]

@blueorangutan package

[blueorangutan]

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10409

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[vishesh92]

@blueorangutan package

[blueorangutan]

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10436

[vishesh92]

@blueorangutan test

[blueorangutan]

@vishesh92 a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian test result (tid-10941) Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7 Total time taken: 61853 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9415-t10941-kvm-centos7.zip Smoke tests completed. 112 look OK, 25 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_role_account_acls_multiple_mgmt_servers	Error	2.32	test_dynamicroles.py
test_query_async_job_result	Error	100.36	test_async_job.py
test_revoke_certificate	Error	0.01	test_certauthority_root.py
test_configure_ha_provider_invalid	Error	0.01	test_hostha_simulator.py
test_configure_ha_provider_valid	Error	0.01	test_hostha_simulator.py
test_ha_configure_enabledisable_across_clusterzones	Error	0.01	test_hostha_simulator.py
test_ha_disable_feature_invalid	Error	0.01	test_hostha_simulator.py
test_ha_enable_feature_invalid	Error	0.01	test_hostha_simulator.py
test_ha_list_providers	Error	0.01	test_hostha_simulator.py
test_ha_multiple_mgmt_server_ownership	Error	0.01	test_hostha_simulator.py
test_ha_verify_fsm_available	Error	0.01	test_hostha_simulator.py
test_ha_verify_fsm_degraded	Error	0.01	test_hostha_simulator.py
test_ha_verify_fsm_fenced	Error	0.01	test_hostha_simulator.py
test_ha_verify_fsm_recovering	Error	0.01	test_hostha_simulator.py
test_hostha_configure_default_driver	Error	0.01	test_hostha_simulator.py
test_hostha_configure_invalid_provider	Error	0.01	test_hostha_simulator.py
test_hostha_disable_feature_valid	Error	0.01	test_hostha_simulator.py
test_hostha_enable_feature_valid	Error	0.01	test_hostha_simulator.py
test_hostha_enable_feature_without_setting_provider	Error	0.01	test_hostha_simulator.py
test_list_ha_for_host	Error	0.01	test_hostha_simulator.py
test_list_ha_for_host_invalid	Error	0.01	test_hostha_simulator.py
test_list_ha_for_host_valid	Error	0.01	test_hostha_simulator.py
test_01_host_ping_on_alert	Error	0.08	test_host_ping.py
test_01_host_ping_on_alert	Error	0.08	test_host_ping.py
test_01_browser_migrate_template	Error	15.36	test_image_store_object_migration.py
test_01_invalid_upgrade_kubernetes_cluster	Failure	223.51	test_kubernetes_clusters.py
test_02_upgrade_kubernetes_cluster	Failure	241.79	test_kubernetes_clusters.py
test_03_deploy_and_scale_kubernetes_cluster	Failure	230.65	test_kubernetes_clusters.py
test_04_autoscale_kubernetes_cluster	Failure	243.46	test_kubernetes_clusters.py
test_05_basic_lifecycle_kubernetes_cluster	Failure	246.97	test_kubernetes_clusters.py
test_06_delete_kubernetes_cluster	Failure	231.45	test_kubernetes_clusters.py
test_08_upgrade_kubernetes_ha_cluster	Failure	341.19	test_kubernetes_clusters.py
test_10_vpc_tier_kubernetes_cluster	Failure	250.78	test_kubernetes_clusters.py
test_11_test_unmanaged_cluster_lifecycle	Error	83.93	test_kubernetes_clusters.py
test_01_add_delete_kubernetes_supported_version	Error	0.12	test_kubernetes_supported_versions.py
login_test_saml_user	Error	3.06	test_login.py
test_01_deployVMInSharedNetwork	Error	82.48	test_network.py
test_03_destroySharedNetwork	Failure	1.08	test_network.py
ContextSuite context=TestSharedNetwork>:teardown	Error	2.18	test_network.py
test_oobm_issue_power_cycle	Error	3.32	test_outofbandmanagement_nestedplugin.py
test_oobm_issue_power_off	Error	3.31	test_outofbandmanagement_nestedplugin.py
test_oobm_issue_power_on	Error	3.29	test_outofbandmanagement_nestedplugin.py
test_oobm_issue_power_reset	Error	3.33	test_outofbandmanagement_nestedplugin.py
test_oobm_issue_power_soft	Error	3.31	test_outofbandmanagement_nestedplugin.py
test_oobm_issue_power_status	Error	2.25	test_outofbandmanagement_nestedplugin.py
test_oobm_background_powerstate_sync	Failure	21.66	test_outofbandmanagement.py
test_oobm_background_powerstate_sync	Error	21.66	test_outofbandmanagement.py
test_oobm_configure_default_driver	Error	0.06	test_outofbandmanagement.py
test_oobm_configure_invalid_driver	Error	0.06	test_outofbandmanagement.py
test_oobm_disable_feature_invalid	Error	0.04	test_outofbandmanagement.py
test_oobm_disable_feature_valid	Error	1.15	test_outofbandmanagement.py
test_oobm_enable_feature_invalid	Error	0.06	test_outofbandmanagement.py
test_oobm_enable_feature_valid	Error	1.13	test_outofbandmanagement.py
test_oobm_enabledisable_across_clusterzones	Error	10.93	test_outofbandmanagement.py
test_oobm_enabledisable_across_clusterzones	Error	10.93	test_outofbandmanagement.py
test_oobm_issue_power_cycle	Error	4.34	test_outofbandmanagement.py
test_oobm_issue_power_cycle	Error	4.34	test_outofbandmanagement.py
test_oobm_issue_power_off	Error	3.36	test_outofbandmanagement.py
test_oobm_issue_power_off	Error	3.36	test_outofbandmanagement.py
test_oobm_issue_power_on	Error	4.36	test_outofbandmanagement.py
test_oobm_issue_power_on	Error	4.36	test_outofbandmanagement.py
test_oobm_issue_power_reset	Error	4.35	test_outofbandmanagement.py
test_oobm_issue_power_reset	Error	4.35	test_outofbandmanagement.py
test_oobm_issue_power_soft	Error	3.33	test_outofbandmanagement.py
test_oobm_issue_power_soft	Error	3.33	test_outofbandmanagement.py
test_oobm_issue_power_status	Error	4.35	test_outofbandmanagement.py
test_oobm_issue_power_status	Error	4.35	test_outofbandmanagement.py
test_oobm_multiple_mgmt_server_ownership	Error	1.16	test_outofbandmanagement.py
test_oobm_multiple_mgmt_server_ownership	Error	1.16	test_outofbandmanagement.py
test_oobm_zchange_password	Error	2.24	test_outofbandmanagement.py
test_oobm_zchange_password	Error	2.24	test_outofbandmanagement.py
test_02_edit_primary_storage_tags	Error	0.02	test_primary_storage.py
test_01_primary_storage_scope_change	Error	0.08	test_primary_storage_scope.py
test_01_vpc_privategw_acl	Error	0.03	test_privategw_acl_ovs_gre.py
test_03_vpc_privategw_restart_vpc_cleanup	Error	0.02	test_privategw_acl_ovs_gre.py
test_05_vpc_privategw_check_interface	Error	0.02	test_privategw_acl_ovs_gre.py
test_01_vpc_privategw_acl	Error	50.60	test_privategw_acl.py
test_02_vpc_privategw_static_routes	Error	200.29	test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup	Error	192.10	test_privategw_acl.py
test_04_rvpc_privategw_static_routes	Error	319.87	test_privategw_acl.py
test_01_purge_expunged_api_vm_start_date	Error	53.95	test_purge_expunged_vms.py
test_02_purge_expunged_api_vm_end_date	Error	51.65	test_purge_expunged_vms.py
test_03_purge_expunged_api_vm_start_end_date	Error	48.39	test_purge_expunged_vms.py
test_04_purge_expunged_api_vm_no_date	Error	45.15	test_purge_expunged_vms.py
test_05_purge_expunged_vm_service_offering	Error	270.12	test_purge_expunged_vms.py
test_06_purge_expunged_vm_background_task	Error	343.30	test_purge_expunged_vms.py
test_01_snapshot_root_disk	Error	3.47	test_snapshots.py
test_CreateTemplateWithDuplicateName	Error	23.92	test_templates.py
test_01_register_template_direct_download_flag	Error	0.15	test_templates.py
test_01_positive_tests_usage	Error	8.34	test_usage_events.py
test_01_ISO_usage	Error	1.08	test_usage.py
test_01_lb_usage	Error	5.24	test_usage.py
test_01_nat_usage	Error	7.31	test_usage.py
test_01_public_ip_usage	Error	1.08	test_usage.py
test_01_snapshot_usage	Error	36.92	test_usage.py
test_01_template_usage	Error	13.52	test_usage.py
test_01_vm_usage	Error	134.86	test_usage.py
test_01_volume_usage	Error	124.86	test_usage.py
test_01_vpn_usage	Error	8.44	test_usage.py
test_04_nonsecured_to_secured_vm_migration	Error	419.33	test_vm_life_cycle.py
test_12_start_vm_multiple_volumes_allocated	Error	10.54	test_vm_life_cycle.py
test_01_vmschedule_create	Error	0.09	test_vm_schedule.py
test_02_redundant_VPC_default_routes	Failure	369.73	test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers	Failure	448.28	test_vpc_redundant.py
test_05_rvpc_multi_tiers	Failure	524.10	test_vpc_redundant.py
test_05_rvpc_multi_tiers	Error	524.11	test_vpc_redundant.py
test_disable_oobm_ha_state_ineligible	Error	0.06	test_hostha_kvm.py
test_hostha_configure_default_driver	Error	0.04	test_hostha_kvm.py
test_hostha_enable_ha_when_host_disabled	Error	0.04	test_hostha_kvm.py
test_hostha_enable_ha_when_host_disconected	Error	0.04	test_hostha_kvm.py
test_hostha_enable_ha_when_host_in_maintenance	Error	0.04	test_hostha_kvm.py
test_hostha_kvm_host_degraded	Error	0.04	test_hostha_kvm.py
test_hostha_kvm_host_fencing	Error	0.04	test_hostha_kvm.py
test_hostha_kvm_host_recovering	Error	0.04	test_hostha_kvm.py
test_remove_ha_provider_not_possible	Error	0.04	test_hostha_kvm.py

[vishesh92]

@blueorangutan package

[blueorangutan]

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[weizhouapache]

@vishesh92 will you move the integration tests to smoke directory ?

[vishesh92]

@vishesh92 will you move the integration tests to smoke directory ?

These tests take too long to run (1-2 hours). It's better to skip them.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10533

[vishesh92]

@blueorangutan test keepEnv

[blueorangutan]

@vishesh92 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

[github-actions[bot]]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[blueorangutan]

[SF] Trillian test result (tid-11034) Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8 Total time taken: 54763 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9415-t11034-kvm-ol8.zip Smoke tests completed. 136 look OK, 1 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_02_unsecure_vm_migration	Error	447.58	test_vm_life_cycle.py
test_03_secured_to_nonsecured_vm_migration	Error	384.12	test_vm_life_cycle.py

[vishesh92]

@blueorangutan package

[blueorangutan]

@vishesh92 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10589

[DaanHoogland]

@blueorangutan package