terraform-provider-proxmox icon indicating copy to clipboard operation
terraform-provider-proxmox copied to clipboard

Soft coding waitForIP

Open sei-gseroka opened this issue 1 year ago • 2 comments

Is your feature request related to a problem? Please describe. When deploying a sys-prepped Windows image with the QEMU guest agent enabled maximum timeout is reached because it has an APIPA. WaitForIP is currently hard coded here.

Describe the solution you'd like Allowing waitForIP to be configurable would cover an edge case where the QEMU guest agent is used to configure a VMs interfaces post deployment.

Describe alternatives you've considered Setting agent/timeout lower than 15 minutes is a workaround, but can lead to false positives causing subsequent provisioning steps to fail.

sei-gseroka avatar Oct 30 '23 16:10 sei-gseroka

WaitForNetworkInterfacesFromVMAgent intentionally waits for IP addresses meeting IsGlobalUnicast criteria. Without it, a link-local IPv6 address would often appear before a DHCP-provided IPv4 address, and VM ipv4_addresses attribute would be returned empty.

Initial provisioning using ssh would then fail, as would other terraform resourced depending on VM's IPv4 address.

It should be easy enough to add an option for the provider to consider any non-loopback ip address good enough, but it seems more like a workaround. The VM at that point would not be fully configured or even accessible.

The IP addresses of VM would be practically unusable (link-local), and all dependent resources could not depend on VM. Instead they would have to depend on some resource that configures VM's networking (null-resource with script or something similar).

It might be more maintainable to have the VM template configure its networking automatically after start. The guest agent would then report final and correct IP address.

For Linux VM cloud-init is the simplest option for statically allocated IP address (assuming DHCP cannot be used).

For Windows VM, there is cloudbase-init, unfortunately it looks like there is some incompatibility thread 1, thread 2 and even requires some patching to make it work with Proxmox "simple" cloud-init configuration.

It looks like the main issue is that Proxmox generates wrong contents of cloud init files, but not the cloud init drive itself. This provider offers control over network_data_file_id, user_data_file_id, vendor_data_file_id, meta_data_file_id docs, it might be possible to actually use cloudbase-init without patching Proxmox, just by generating correct files in terraform. And like linux's cloud-init it could be used to configure authorized SSH keys, etc... (If you try this route and succeed, consider adding an example to the provider documentation, it could help others)

Another option is to borrow cloud-init's use of smbios serial field to store additional data.

Place networking configuration in smbios.serial field in VM's terraform configuration, and inside the VM have an autostart powershell script read the field and configure the networking using that information. There does not seem to be a fixed length limit to serial field, so it should be enough to pass all networking information like IP, mask, gateway and DNS servers.

There is also the network_device.mac_address field of the provider. If the link local IPv4 address is not intentional, but just a byproduct of a DHCP server assigning IP addresses only to known mac addresses, then the simplest solution could be to just generate some MAC addresses in advance, add them to DHCP server, and then use them for VM's terraform configuration.

otopetrik avatar Oct 30 '23 18:10 otopetrik

Thanks for the response and ideas @otopetrik.

I agree that in most cases waitForIP being set to true is the best option and should remain the default, however my use case only needs to validate that the guest agent is responding. I tried cloudbase-init and ran into the issues you identified, which led me to discover I can talk to the guest agent directly.

For simplicity sake I prefer to store my startup scripts in my terraform repository and then use terraform local-exec to call the guest agent api and use file-write/exec to provision new windows vms. For additional context the vms are being deployed to an isolated network and the terraform server will be unable to reach them by IP regardless of how it is set.

sei-gseroka avatar Oct 31 '23 15:10 sei-gseroka

Marking this issue as stale due to inactivity in the past 180 days. This helps us focus on the active issues. If this issue is reproducible with the latest version of the provider, please comment. If this issue receives no comments in the next 30 days it will automatically be closed. If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

bpg-autobot[bot] avatar Apr 29 '24 00:04 bpg-autobot[bot]