vcenter_vmtemplate_libraryitems: clone VM Template incomplete example and no cloud-init example

[sean-freeman]

SUMMARY

Documentation of vcenter_vmtemplate_libraryitems contains incomplete example, and no cloud-init example

It should be expected if automation is used, that a clear example of a VM Template clone is provided in the documentation (especially given vcenter_vm Ansible Module can only use source as Instance Clone or Clone of a VM, and can not use a VM Template).

ISSUE TYPE

• Documentation Report

COMPONENT NAME

vcenter_vmtemplate_libraryitems

ANSIBLE VERSION

• All

---

PROPOSED SOLUTION

The following is a working example, although the cloud-init via configuration specification does not appear to work (even though it follows the REST API Specification, as shown with code comments).

The VM Template must be prepared with cloud-init:

• Edit /etc/cloud/cloud.cfg with data source for VMware (and not OVF).

  # Enable VMware VM Guest OS Customization with cloud-init (set to true for traditional customization)
  disable_vmware_customization: false
  
  # Use allow raw data to directly use the cloud-init metadata and user data files provided by the VMware VM Customization Specification
  # Wait 120 seconds for VMware VM Customization file to be available
  datasource:
    VMware:
      allow_raw_data: true
      vmware_cust_file_max_wait: 60
  

• Run:
  • vmware-toolbox-cmd config set deployPkg enable-custom-scripts true
  • vmware-toolbox-cmd config set deployPkg wait-cloudinit-timeout 60
  • sudo cloud-init clean --seed --logs to remove cloud-init logs, remove cloud-init seed directory /var/lib/cloud/seed.
    • If using cloud-init versions prior to 22.3.0 then do not use --machine-id parameter.
    • The --machine-id parameter which removes /etc/machine-id may on first reboot cause the OS Network Interfaces to be DOWN which causes the DHCP Request to silently error.
• Shutdown, then Clone as Template to Library

Ansible Playbook example:

- name: Define variables
  ansible.builtin.set_fact:
    vmware_vm_folder_name: ""
    vmware_vm_cluster_name: ""
    vmware_vm_cluster_host_name: ""
    vmware_vm_cluster_datastore_name: ""
    vmware_vm_content_library_name: ""

    vmware_vm_target_name: ""
    vmware_vm_dns_root_domain: ""
    vmware_vm_ssh_public_key_file_path: ""


- name: Identify VM Folder
  register: register_vmware_vm_folder
  vmware.vmware_rest.vcenter_folder_info:
    names: "{{ vmware_vm_folder_name }}"
    type: VIRTUAL_MACHINE

- name: Identify Datacenter Cluster
  register: register_vmware_vm_cluster
  vmware.vmware_rest.vcenter_cluster_info:
    names: "{{ vmware_vm_cluster_name }}"

- name: Identify Host in Datacenter Cluster
  register: register_vmware_vm_cluster_host
  vmware.vmware_rest.vcenter_host_info:
    names: "{{ vmware_vm_cluster_host_name }}"

- name: Identify Datastore
  register: register_vmware_vm_cluster_datastore
  vmware.vmware_rest.vcenter_datastore_info:
    names: "{{ vmware_vm_cluster_datastore_name }}"

- name: Identify Content Library (to store VM Template)
  register: register_vmware_vm_content_library
  vmware.vmware_rest.content_locallibrary:
    name: "{{ vmware_vm_content_library_name }}"

- name: List all items in Content Library
  register: register_vmware_vm_content_library_items
  vmware.vmware_rest.content_library_item_info:
    library_id: "{{ register_vmware_vm_content_library.id }}"

- name: Identify VMware Template ID
  ansible.builtin.set_fact:
    vmware_vm_template_id: "{{ (register_vmware_vm_content_library_items.value | selectattr('type', '==', 'vm-template') | selectattr('name', '==', vmware_vm_template_name) | first).id }}"


- name: Check if VM exists
  register: register_check_vm_exists
  vmware.vmware_rest.vcenter_vm_info:
    names: "{{ vmware_vm_target_name }}"

# If VM exists
- name: Set VM ID
  when: not register_check_vm_exists.value | length == 0
  ansible.builtin.set_fact:
    register_vmware_vm_cluster_host_id: "{{ register_check_vm_exists.value[0].vm }}" # VM ID

- name: Check VM status
  register: register_provisioned_host_single_info
  when: not register_check_vm_exists.value | length == 0
  vmware.vmware_rest.vcenter_vm:
    vm: "{{ register_vmware_vm_cluster_host_id }}" # VM ID


# Deploy a Virtual Machine from a VM Template in a Content Library
# Doc: https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.vm_admin.doc/GUID-6EA309BC-9113-449C-B668-ACBB363485C3.html
- name: Provision VMware Virtual Machine based upon the VM Template
  register: register_provisioned_host_single
  vmware.vmware_rest.vcenter_vmtemplate_libraryitems:

    placement:
      folder: "{{ (register_vmware_vm_folder.value | first).folder }}"
      # resource_pool: ""
      cluster: "{{ (register_vmware_vm_cluster.value | first).cluster }}"
      host: "{{ (register_vmware_vm_cluster_host.value | first).host }}"

    template_library_item: '{{ vmware_vm_template_id }}' # ID of the Content Library Item with the source VM Template (not OVF) to be cloned and deployed
    state: deploy # Deploy the VM Template defined in template_library_item
    powered_on: false # false when customization specification with cloud-init is required
    session_timeout: 600 # 10 minutes

    name: "{{ vmware_vm_target_name }}"
    description: "{{ vmware_vm_target_name }} created by Ansible"

    # May cause conflict with powered_on parameter
    hardware_customization:
      cpu_update:
        num_cpus: 2
        num_cores_per_socket: 2
      memory_update:
        memory: "{{ 16 * 1024 }}" # MiB
      # nics:

    # Boot Disk will be loaded to this datastore
    disk_storage:
      datastore: "{{ (register_vmware_vm_cluster_datastore.value | first).datastore }}"
      # storage_policy:


# After VM is provisioned, does not return value if previously provisioned
- name: Set VM ID
  when: register_check_vm_exists.value | length == 0
  ansible.builtin.set_fact:
    register_vmware_vm_cluster_host_id: "{{ register_provisioned_host_single.value }}" # Returned from VM provision

- name: Check VM status
  register: register_provisioned_host_single_info
  vmware.vmware_rest.vcenter_vm:
    vm: "{{ register_vmware_vm_cluster_host_id }}"


# Example https://cloudinit.readthedocs.io/en/23.4.1/reference/datasources/vmware.html#walkthrough-of-guestinfo-keys-transport
# Docs https://developer.vmware.com/docs/18555/GUID-75E27FA9-2E40-4CBF-BF3D-22DCFC8F11F7.html
# >> The instance-id key is required. All other keys are optional.
- name: Set cloud-init variables for customization specification
  when: register_provisioned_host_single_info.value.power_state is defined and register_provisioned_host_single_info.value.power_state != "POWERED_ON"
  ansible.builtin.set_fact:
    metadata_yaml:
      instance-id: "{{ vmware_vm_target_name }}"
      hostname: "{{ vmware_vm_target_name }}"
      local-hostname: "{{ vmware_vm_target_name }}"
      network:
        version: 2
        ethernets:
          nics:
            match:
              name: e*
            dhcp4: true
            dhcp6: false
      public_ssh_keys:
        - "{{ lookup('ansible.builtin.file', vmware_vm_ssh_public_key_file_path) }}"

    userdata_yaml_text: |
      #cloud-config

      hostname: {{ vmware_vm_target_name }}
      fqdn: {{ vmware_vm_target_name }}.{{ vmware_vm_dns_root_domain }}

      # timezone: "Etc/UTC"

      disable_root: false

      ssh_pwauth: false

      ssh_deletekeys: true

      ssh:
        emit_keys_to_console: false

      no_ssh_fingerprints: false

      ssh_authorized_keys:
        - {{ lookup('ansible.builtin.file', vmware_vm_ssh_public_key_file_path) }}

      users:
        - name: root
          ssh_authorized_keys:
            - {{ lookup('ansible.builtin.file', vmware_vm_ssh_public_key_file_path) }}
          lock_passwd: false

      write_files:
        - path: /etc/cloud/cloud-init.disabled
          permissions: "0644"
          content: ""


# Doc 1 https://developer.vmware.com/apis/vsphere-automation/latest/vcenter/api/vcenter/vm/vm/guest/customization/put/
# Doc 2 https://developer.vmware.com/docs/18555/GUID-75E27FA9-2E40-4CBF-BF3D-22DCFC8F11F7.html
# >> cloud-init required: metadata as plain-text JSON/YAML, maximum 512KB file size
# >> cloud-init optional: userdata as plain-text in raw cloud-init format with no compression / no base64 encoding, maximum 512KB file size
# Error 400 com.vmware.vapi.std.errors.not_allowed_in_current_state : if the virtual machine vm is not in a powered off state.
- name: Apply customization specification to the VM in Powered Off state
  when: register_provisioned_host_single_info.value.power_state is defined and register_provisioned_host_single_info.value.power_state != "POWERED_ON"
  vmware.vmware_rest.vcenter_vm_guest_customization:
    vm: '{{ register_vmware_vm_cluster_host_id }}'
    configuration_spec:
      cloud_config:
        type: CLOUDINIT
        cloudinit:
          metadata: "{{ metadata_yaml | to_json(ensure_ascii=true) }}"
          userdata: "{{ userdata_yaml_text | trim }}" # remove last newline character
      # linux_config:
    interfaces: []
    global_DNS_settings: {}


- name: Ensure VM is Powered ON
  register: register_vm_power_info
  vmware.vmware_rest.vcenter_vm_power:
    state: start
    vm: "{{ register_vmware_vm_cluster_host_id }}"
  # Wait until VM is powered on
  until: (register_vm_power_info.value.error_type is defined and register_vm_power_info.value.error_type == "ALREADY_IN_DESIRED_STATE")
  retries: 15
  delay: 60

- name: Show VM Information
  register: register_vm_info
  vmware.vmware_rest.vcenter_vm_info:
    vm: '{{ register_vmware_vm_cluster_host_id }}'
  # Wait until VM is powered on
  until: register_vm_info.value.power_state == "POWERED_ON"
  retries: 45
  delay: 20

- name: Get guest networking information (wait until DHCP assigns IP Address for host)
  register: register_vm_nic_info
  vmware.vmware_rest.vcenter_vm_guest_networking_interfaces_info:
    vm: '{{ register_vmware_vm_cluster_host_id }}'
  # Wait until VM Tools is running
  until: (register_vm_nic_info.value.error_type | default("")) != "SERVICE_UNAVAILABLE" and (register_vm_nic_info.value[0].ip.ip_addresses | length) > 0
  retries: 45
  delay: 20

[sean-freeman]

@ephracis This should be given some attention, cloning from a VM Template is a critical functionality any administrator would expect

[ephracis]

Hm… I could take some of the code I wrote for my current customer and use it as an example.

But they don’t use cloud init (unfortunately, if you ask me) so I don’t have a working example of that.

I’ll try to get something up next week, but I can’t promise anything since my main work is consulting and not maintaining this module. But for you I’ll make an effort. ☺️

[sean-freeman]

@ephracis Appreciated greatly 🙂 Is there nobody from Broadcom that is looking at this Ansible Module too?

The "VMware VM Guest OS Customization with cloud-init" execution is very fragile from my testing experience, I could never get 'userdata' to work 100% of the time. It would need an expert to debug why the 'userdata' does not always execute.

[sean-freeman]

Maybe @goneri has a contact in Broadcom who could assist the cloud-init 'userdata' debugging?

[ephracis]

I just checked and the code I wrote for the customer is pretty much the same as the last three tasks in the example (get items, find template, deploy VM). So my code will not add much to the existing example.

[sean-freeman]

@GomathiselviS @alinabuzachis @jillr @hakbailey @gravesm Who is the owner of this Ansible Collection and responsible for testing in collaboration with VMware?

It's not right to have the Ansible Collection failing when used with cloud-init and with so many outdated examples in the documentation; the end-user should expect such functionality to work and documentation to be accurate.

[jillr]

Hi @sean-freeman This collection is developed by the Ansible Content Engineering organization, there is no one from Broadcom/VMware involved in this collection. Thank you for the bug report and contributing to the improvement of this collection.

[sean-freeman]

@jillr In order to solve the poor documentation and debug the vcenter_vm_guest_customization not working as expected with cloud-init, a VMware expert will be required.

[sean-freeman]

Code has been taken from this GH Issue and added into the Ansible Collection via PR #534 , even though the procedure is still broken for cloud-init and all end-users will suffer failures.

[sean-freeman]

@mikemorency PR https://github.com/ansible-collections/vmware.vmware_rest/pull/534 copied the cloud-init provided above; which I had extrapolated from reading a lot of documention..... but I could not get working.

In Issue #527 you stated this code worked but did not state what versions of vSphere/vCenter/VCF were used.

However, I can see the flow appears different than what an admin (inc. myself) would expect. Can you please confirm below.....

Ansible Playbook example provided in this GH Issue performs:

• vcenter_vmtemplate_libraryitems, with state: deploy and powered_on: false. Return VM ID of deployed VM Template
• vcenter_vm_guest_customization, with cloud-init specification
• vcenter_vm_power, with state: start
• :x: fail

In 534 / current documentation, your flow is:

• vcenter_vmtemplate_libraryitems, with state:deploy only. Return VM ID of deployed VM Template
• vcenter_vm_power, with state: start
• vcenter_vm_tools_info
• vcenter_vm_power, with state: stop
• vcenter_vm_guest_customization, with cloud-init specification
• I assume missing final Ansible Task, is vcenter_vm_power to start
• ✅ success

ASK: Your documentation implies a VM Template must be cloned and started, then stopped and cloud-init subsequently run?

The module's documentation alludes to this with:

# 3. VMware tools must be installed and recognized by vCenter before you can apply customization. See the example below for one approach to this.

But it appears, there is only this "Clone > VM On > VM Off > Customization Spec > VM On" approach?

[mikemorency]

Hi @sean-freeman

there is only this "Clone > VM On > VM Off > Customization Spec > VM On" approach?

No, this approach is only required if VMware tools is not installed and registered on the template. If the template has VMware tools and vCenter shows it's version, then you can use the workflow you are accustomed to: Clone > Customize > VM on

I hope this helps. I will update the documentation to be more specific about start/stop tasks (and add the missing final power on)

[sean-freeman]

@mikemorency I will commence a test, I could never get cloud-init working previously (on RHEL 8/9 or SLES 12/15)

[mikemorency]

Hi, I am going to go ahead an close this given the updated docs and inactivity. Please feel free to continue to comment, we can re-open if needed