cloudstack icon indicating copy to clipboard operation
cloudstack copied to clipboard

CKS: add ConfigDrive to cloud-init datasource_list in systemvm template

Open weizhouapache opened this issue 2 years ago • 56 comments

Description

This PR fixes #7647

Types of changes

  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [ ] New feature (non-breaking change which adds functionality)
  • [x] Bug fix (non-breaking change which fixes an issue)
  • [ ] Enhancement (improves an existing feature and functionality)
  • [ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • [ ] Major
  • [ ] Minor

Bug Severity

  • [ ] BLOCKER
  • [ ] Critical
  • [ ] Major
  • [ ] Minor
  • [ ] Trivial

Screenshots (if appropriate):

How Has This Been Tested?

weizhouapache avatar Jun 19 '23 12:06 weizhouapache

This requires a new systemvm template cc @kiranchavala

weizhouapache avatar Jun 19 '23 12:06 weizhouapache

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 31.06%. Comparing base (6dc3d06) to head (7fce265).

Additional details and impacted files
@@              Coverage Diff              @@
##               main    #7650       +/-   ##
=============================================
+ Coverage     15.88%   31.06%   +15.18%     
- Complexity    15718    33881    +18163     
=============================================
  Files          5172     5397      +225     
  Lines        364426   379397    +14971     
  Branches      53574    55363     +1789     
=============================================
+ Hits          57874   117855    +59981     
+ Misses       299648   245905    -53743     
- Partials       6904    15637     +8733     
Flag Coverage Δ
simulator-marvin-tests 24.64% <ø> (?)
uitests 4.34% <ø> (ø)
unit-tests 16.87% <ø> (+<0.01%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Jun 19 '23 13:06 codecov[bot]

@blueorangutan package

weizhouapache avatar Jun 19 '23 17:06 weizhouapache

@weizhouapache a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Jun 19 '23 17:06 blueorangutan

Packaging result [SF]: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: el9 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 6269

blueorangutan avatar Jun 19 '23 18:06 blueorangutan

@blueorangutan test

weizhouapache avatar Jun 19 '23 19:06 weizhouapache

@weizhouapache a [SF] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan avatar Jun 19 '23 19:06 blueorangutan

[SF] Trillian test result (tid-6772) Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7 Total time taken: 57436 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7650-t6772-kvm-centos7.zip Smoke tests completed. 107 look OK, 1 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_01_migrate_VM_and_root_volume Error 82.72 test_vm_life_cycle.py
test_02_migrate_VM_with_two_data_disks Error 52.45 test_vm_life_cycle.py

blueorangutan avatar Jun 20 '23 11:06 blueorangutan

[SF] Trillian test result (tid-6837) Environment: kvm-rocky8 (x2), Advanced Networking with Mgmt server r8 Total time taken: 41197 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7650-t6837-kvm-rocky8.zip Smoke tests completed. 108 look OK, 0 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File

blueorangutan avatar Jun 25 '23 06:06 blueorangutan

[SF] Trillian test result (tid-6839) Environment: vmware-70u3 (x2), Advanced Networking with Mgmt server 7 Total time taken: 42825 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7650-t6839-vmware-70u3.zip Smoke tests completed. 108 look OK, 0 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File

blueorangutan avatar Jun 25 '23 06:06 blueorangutan

[SF] Trillian test result (tid-6838) Environment: kvm-ubuntu22 (x2), Advanced Networking with Mgmt server u22 Total time taken: 45885 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7650-t6838-kvm-ubuntu22.zip Smoke tests completed. 108 look OK, 0 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File

blueorangutan avatar Jun 25 '23 07:06 blueorangutan

[SF] Trillian test result (tid-6840) Environment: xcpng82 (x2), Advanced Networking with Mgmt server 7 Total time taken: 66601 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7650-t6840-xcpng82.zip Smoke tests completed. 104 look OK, 4 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_01_non_strict_host_anti_affinity Error 197.66 test_nonstrict_affinity_group.py
test_02_non_strict_host_affinity Error 130.22 test_nonstrict_affinity_group.py
test_deploy_vm_with_registered_userdata Error 801.24 test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_allow Error 819.10 test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_append Error 823.81 test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_params Error 820.90 test_register_userdata.py
test_01_internallb_roundrobin_1VPC_3VM_HTTP_port80 Failure 595.31 test_internal_lb.py
test_list_volumes_metrics_history Failure 188.35 test_metrics_api.py

blueorangutan avatar Jun 25 '23 13:06 blueorangutan

thanks @rohityadavcloud @DaanHoogland for review

I have kicked some trillian test using the new templates. The kvm/vmware test results (see previous comments) look good. There were some test failures with xcpng82. I have rekick the trillian test

weizhouapache avatar Jun 26 '23 08:06 weizhouapache

[SF] Trillian test result (tid-6841) Environment: xcpng82 (x2), Advanced Networking with Mgmt server r8 Total time taken: 70964 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7650-t6841-xcpng82.zip Smoke tests completed. 105 look OK, 3 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_01_non_strict_host_anti_affinity Error 198.92 test_nonstrict_affinity_group.py
test_02_non_strict_host_affinity Error 110.32 test_nonstrict_affinity_group.py
test_deploy_vm_with_registered_userdata Error 825.22 test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_allow Error 862.03 test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_append Error 850.41 test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_params Error 839.74 test_register_userdata.py
test_list_volumes_metrics_history Failure 284.39 test_metrics_api.py

blueorangutan avatar Jun 26 '23 13:06 blueorangutan

[SF] Trillian Build Failed (tid-6859)

blueorangutan avatar Jun 26 '23 14:06 blueorangutan

[SF] Trillian test result (tid-6861) Environment: vmware-80 (x2), Advanced Networking with Mgmt server 7 Total time taken: 62348 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7650-t6861-vmware-80.zip Smoke tests completed. 105 look OK, 3 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_03_migration_options_storage_tags Error 31.72 test_primary_storage.py
test_01_deploy_vm_on_specific_host Error 21.67 test_vm_deployment_planner.py
test_02_deploy_vm_on_specific_cluster Error 3601.65 test_vm_deployment_planner.py
test_03_deploy_vm_on_specific_pod Error 1.24 test_vm_deployment_planner.py
test_04_deploy_vm_on_host_override_pod_and_cluster Error 2.24 test_vm_deployment_planner.py
test_05_deploy_vm_on_cluster_override_pod Error 2.24 test_vm_deployment_planner.py
test_01_offline_migrate_VM_and_root_volume Error 122.79 test_vm_life_cycle.py
test_02_offline_migrate_VM_with_two_data_disks Error 67.68 test_vm_life_cycle.py
test_03_live_migrate_VM_with_two_data_disks Error 65.84 test_vm_life_cycle.py

blueorangutan avatar Jun 27 '23 07:06 blueorangutan

[SF] Trillian test result (tid-6858) Environment: xcpng81 (x2), Advanced Networking with Mgmt server r8 Total time taken: 70965 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7650-t6858-xcpng81.zip Smoke tests completed. 104 look OK, 4 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_01_non_strict_host_anti_affinity Error 208.03 test_nonstrict_affinity_group.py
test_02_non_strict_host_affinity Error 119.55 test_nonstrict_affinity_group.py
test_deploy_vm_with_registered_userdata Error 822.24 test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_allow Error 830.59 test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_append Error 835.42 test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_params Error 849.77 test_register_userdata.py
test_list_vms_metrics_admin Error 0.23 test_metrics_api.py
test_list_vms_metrics_history Error 0.22 test_metrics_api.py
test_list_volumes_metrics_history Error 0.22 test_metrics_api.py
test_01_deploy_vm_on_specific_host Error 0.11 test_vm_deployment_planner.py
test_02_deploy_vm_on_specific_cluster Error 0.09 test_vm_deployment_planner.py
test_03_deploy_vm_on_specific_pod Error 0.13 test_vm_deployment_planner.py
test_04_deploy_vm_on_host_override_pod_and_cluster Error 0.14 test_vm_deployment_planner.py
test_05_deploy_vm_on_cluster_override_pod Error 0.10 test_vm_deployment_planner.py

blueorangutan avatar Jun 27 '23 10:06 blueorangutan

[SF] Trillian test result (tid-6860) Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7 Total time taken: 77688 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr7650-t6860-vmware-67u3.zip Smoke tests completed. 104 look OK, 4 have errors, 0 did not run Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_04_rvpc_privategw_static_routes Failure 621.51 test_privategw_acl.py
test_provision_certificate Error 61.46 test_certauthority_root.py
test_01_create_template Error 896.41 test_templates.py
test_03_create_template_with_checksum_sha256 Error 65.47 test_templates.py
test_03_delete_template Error 824.08 test_templates.py
test_04_extract_template Error 1.07 test_templates.py
test_deployvm_userdispersing Error 1079.74 test_deploy_vms_with_varied_deploymentplanners.py

blueorangutan avatar Jun 27 '23 11:06 blueorangutan

@blueorangutan package

DaanHoogland avatar Jul 03 '23 09:07 DaanHoogland

@DaanHoogland a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Jul 03 '23 09:07 blueorangutan

Packaging result [SF]: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: el9 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 6391

blueorangutan avatar Jul 03 '23 10:07 blueorangutan

@blueorangutan package

weizhouapache avatar Jul 24 '23 01:07 weizhouapache

this is on hold, because multiple datasources are not well-supported by cloud-init the new systemvm template works well on kvm, but not on vmware/xen

blocked by https://github.com/canonical/cloud-init/pull/4281

weizhouapache avatar Jul 24 '23 01:07 weizhouapache

@weizhouapache a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Jul 24 '23 01:07 blueorangutan

Packaging result [SF]: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: el9 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 6540

blueorangutan avatar Jul 24 '23 02:07 blueorangutan

@blueorangutan package

weizhouapache avatar Aug 11 '23 13:08 weizhouapache

@weizhouapache a [SF] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan avatar Aug 11 '23 13:08 blueorangutan

Packaging result [SF]: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: el9 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 6753

blueorangutan avatar Aug 11 '23 14:08 blueorangutan

@blueorangutan test rocky8 kvm-rocky8 keepEnv

weizhouapache avatar Aug 11 '23 15:08 weizhouapache

@weizhouapache a [SF] Trillian-Jenkins test job (rocky8 mgmt + kvm-rocky8) has been kicked to run smoke tests

blueorangutan avatar Aug 11 '23 15:08 blueorangutan