Add logs to VMware instance configuration process

[winterhazel]

Description

The virtual machine configuration process is used in a lot of VMware workflows, such as instance creation, start and scale. However, there are no logs in this process, which difficults the troubleshooting of errors, specially of those that are returned by VMware.

This PR adds logs to this process in order to make the troubleshooting of errors easier.

Types of changes

• [ ] Breaking change (fix or feature that would cause existing functionality to change)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Bug fix (non-breaking change which fixes an issue)
• [X] Enhancement (improves an existing feature and functionality)
• [ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• [ ] Major
• [X] Minor

Screenshots (if appropriate):

How Has This Been Tested?

In a local lab:

1. I created a VM using only IDE controllers;
2. I stopped the VM;
3. I changed the controllers of the VM from IDE to SCSI;
4. I started the VM, and verified that the logs presented the reconfigurations of the VM in order to remove the disks and NICs, to add the SCSI controllers, and the general VM configuration.

ubuntu@lab-vmware-mgmt-01:~$ sudo grep 'Reconfiguring virtual machine' $mgmt_log
2024-05-14T11:17:15,122 DEBUG [c.c.h.v.m.VirtualMachineMO] (DirectAgent-147:[ctx-9fd2c09e, 172.16.75.21, job-783/job-784, cmd: StartCommand]) (logid:64cbc567) Reconfiguring virtual machine [{"internalCSName":"i-2-217-VM"}] using spec [{"deviceChange":[{"operation":"REMOVE","device":{"capacityInKB":51200,"capacityInBytes":52428800,"shares":{"shares":1000,"level":"NORMAL"},"storageIOAllocation":{"limit":-1,"shares":{"shares":1000,"level":"NORMAL"},"reservation":0},"diskObjectId":"518-3001","key":3001,"deviceInfo":{"label":"Hard disk 1","summary":"51,200 KB"},"backing":{"diskMode":"persistent","split":false,"writeThrough":false,"thinProvisioned":true,"uuid":"6000C29b-5522-5947-c391-e29f44c11a73","contentId":"fb640b892cf0174da714e57339203aa5","digestEnabled":false,"sharing":"sharingNone","fileName":"[c2b40d03a7b132fc928f7f9f39cf4297] i-2-217-VM/ROOT-217.vmdk","datastore":{"value":"datastore-22","type":"Datastore"},"backingObjectId":""},"controllerKey":200,"unitNumber":1}},{"operation":"REMOVE","device":{"addressType":"manual","macAddress":"02:01:00:d0:00:04","wakeOnLanEnabled":true,"resourceAllocation":{"reservation":0,"share":{"shares":50,"level":"NORMAL"},"limit":-1},"uptCompatibilityEnabled":false,"key":4000,"deviceInfo":{"label":"Network adapter 1","summary":"cloud.guest.615.200.1-vSwitch0"},"backing":{"network":{"value":"network-380","type":"Network"},"deviceName":"cloud.guest.615.200.1-vSwitch0","useAutoDetect":false},"connectable":{"startConnected":true,"allowGuestControl":true,"connected":false,"status":"untried"},"slotInfo":{"pciSlotNumber":32},"controllerKey":100,"unitNumber":7}}]}].
2024-05-14T11:17:16,964 DEBUG [c.c.h.v.m.VirtualMachineMO] (DirectAgent-147:[ctx-9fd2c09e, 172.16.75.21, job-783/job-784, cmd: StartCommand]) (logid:64cbc567) Reconfiguring virtual machine [{"internalCSName":"i-2-217-VM"}] using spec [{"deviceChange":[{"operation":"ADD","device":{"sharedBus":"NO_SHARING","busNumber":0,"key":-4}},{"operation":"ADD","device":{"sharedBus":"NO_SHARING","busNumber":1,"key":-3}},{"operation":"ADD","device":{"sharedBus":"NO_SHARING","busNumber":2,"key":-2}},{"operation":"ADD","device":{"sharedBus":"NO_SHARING","busNumber":3,"key":-1}}]}].
2024-05-14T11:17:29,603 DEBUG [c.c.h.v.m.VirtualMachineMO] (DirectAgent-149:[ctx-a1985c08, 172.16.75.21, job-783/job-785, cmd: StartCommand]) (logid:64cbc567) Reconfiguring virtual machine [{"internalCSName":"i-2-217-VM"}] using spec [{"guestId":"otherLinux64Guest","numCPUs":1,"memoryMB":512,"memoryHotAddEnabled":false,"cpuHotAddEnabled":false,"deviceChange":[{"operation":"EDIT","device":{"key":3000,"deviceInfo":{"label":"CD/DVD drive 1","summary":"Remote device"},"backing":{"exclusive":false,"deviceName":""},"connectable":{"startConnected":true,"allowGuestControl":false,"connected":true},"controllerKey":200,"unitNumber":0}},{"operation":"ADD","device":{"capacityInKB":0,"key":-2,"backing":{"diskMode":"persistent","fileName":"[c2b40d03a7b132fc928f7f9f39cf4297] i-2-217-VM/ROOT-217.vmdk","datastore":{"value":"datastore-22","type":"Datastore"}},"connectable":{"startConnected":true,"allowGuestControl":false,"connected":true},"controllerKey":1000,"unitNumber":0}},{"operation":"ADD","device":{"addressType":"Manual","macAddress":"02:01:00:d0:00:04","key":-3,"backing":{"network":{"value":"network-380","type":"Network"},"deviceName":"cloud.guest.615.200.1-vSwitch0"},"connectable":{"startConnected":true,"allowGuestControl":true,"connected":true}}}],"cpuAllocation":{"reservation":0,"limit":-1},"memoryAllocation":{"reservation":0,"limit":512},"extraConfig":[{"key":"machine.id","value":""},{"key":"devices.hotplug","value":"true"},{"key":"nvp.vm-uuid","value":"ee781edb-bb7b-4d15-8dfe-88b582bc1419"},{"key":"nvp.iface-id.0","value":"47973c05-5956-4c89-a0d8-b7ef218cb7de"},{"key":"dataDiskController","value":"scsi"},{"key":"Message.ReservedCapacityFreed.Flag","value":"false"},{"key":"nestedVirtualizationFlag","value":"false"},{"key":"vmware.reserve.mem","value":"false"},{"key":"vmware.reserve.cpu","value":"false"},{"key":"nicAdapter","value":"E1000"},{"key":"rootDiskController","value":"scsi"},{"key":"RemoteDisplay.vnc.enabled","value":"true"},{"key":"RemoteDisplay.vnc.password","value":"xDys93MKuuAJTnvoPya1tw"},{"key":"RemoteDisplay.vnc.port","value":"5918"}]}].

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 4.25%. Comparing base (21af134) to head (117b0a2). Report is 552 commits behind head on main.

:exclamation: There is a different number of reports uploaded between BASE (21af134) and HEAD (117b0a2). Click for more details.

HEAD has 1 upload less than BASE

Flag	BASE (21af134)	HEAD (117b0a2)
unittests	1	0

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #9080       +/-   ##
============================================
- Coverage     15.28%   4.25%   -11.04%     
============================================
  Files          5425     363     -5062     
  Lines        474138   29686   -444452     
  Branches      58984    5214    -53770     
============================================
- Hits          72486    1264    -71222     
+ Misses       393584   28279   -365305     
+ Partials       8068     143     -7925     

Flag	Coverage Δ
uitests	4.25% <ø> (-0.02%)	:arrow_down:
unittests	?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[winterhazel]

@blueorangutan package

[blueorangutan]

@winterhazel a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9613

[winterhazel]

@blueorangutan package

[blueorangutan]

@winterhazel a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 9732

[DaanHoogland]

Did you test @hsato03 ?

[winterhazel]

clgtm in principal. Did you check if this object is not toString()d anywhere else, @winterhazel ?

@DaanHoogland yes, its toString() is not used anywhere, as its not even implemented. Whenever we need to log this object, we're converting it to JSON.

[DaanHoogland]

clgtm in principal. Did you check if this object is not toString()d anywhere else, @winterhazel ?

@DaanHoogland yes, its toString() is not used anywhere, as its not even implemented. Whenever we need to log this object, we're converting it to JSON.

ok, so no default or inherrited toString used anywhere? great.

[JoaoJandre]

@blueorangutan package

[blueorangutan]

@JoaoJandre a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10454

[winterhazel]

@DaanHoogland @rohityadavcloud can we run the CI?

[DaanHoogland]

@blueorangutan test alma8 vmware-70u3

[blueorangutan]

@DaanHoogland a [SL] Trillian-Jenkins test job (alma8 mgmt + vmware-70u3) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian test result (tid-11425) Environment: vmware-70u3 (x2), Advanced Networking with Mgmt server a8 Total time taken: 51027 seconds Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9080-t11425-vmware-70u3.zip Smoke tests completed. 135 look OK, 2 have errors, 0 did not run Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_06_purge_expunged_vm_background_task	Failure	348.12	test_purge_expunged_vms.py
test_02_restore_vm_with_disk_offering	Error	63.32	test_restore_vm.py
test_03_restore_vm_with_disk_offering_custom_size	Error	58.23	test_restore_vm.py