VM stuck at cloud-init while running "init-local"

[EsharkyTheGreat]

Overview of the Issue

I was following along this video to create a ubuntu VM template with cloud-init for proxmox, I copied the packer file and ran it but once the VM starts and the boot command is entered it doesnt proceed further than some cloud-init process

I even tried changing the ISO to 22.04 and used this config

It gets stuck at the cloud-init stage again

Reproduction Steps

Ran packer on the config provided in both of these sources

Plugin and Packer version

packer version 1.8 Plugin proxmox

Simplified Packer Buildfile

https://github.com/justin-p/packer-proxmox-ubuntu2004/blob/main/ubuntu2004.pkr.hcl https://github.com/aerialls/madalynn-packer/blob/master/ubuntu-22.04/ubuntu.json https://github.com/xcad2k/boilerplates/tree/main/packer/proxmox/ubuntu-server-focal-docker

Operating system and Environment details

Proxmox Virtual Environment 6.4-4

Log Fragments and crash.log files

[mabeett]

Hello, The crashlog you are sharing indicates QEMU Guest agent is not running. Is the ISO system or your setup config enabling/installing the agent ? You may build with -debug flag in order to see step by step what is happening and on thisway you could check the proxmox webUI, the Console for the VM and other information

[goffinf]

Quite an old version of Proxmox. I am on 7.2-4. Probably not critical but worth keeping a bit closer to the latest updates IMO.

You haven't said what platform you are using to run Packer. This may be important especially if you are trying to do so on WSL2. You need some specific additional configuration on WSL2 to route requests to the HTTP server Packer creates. Personally I run Packer on a prepared Ubuntu 20.04 VM running in Proxmox, the image for which I create with Packer itself (turtles all the way :-) ). That just makes life easier because Packer's HTTP server will just work OOTB which is a common area where builds fail.

Ensure your cloud-init (./http/user-data) file installs, enables and starts the qemu-guest-agent, otherwise Packer will never be able to obtain the IP address to connect to. Even if you are already doing this (its hard to tell because you haven't shown the contents of your user-data), its probable you have an error in your cloud-config and that causes it not to complete successfully in which case it will just sit there indefinitely. Also worth giving your SSH timeout plenty of time to connect since it can take quite some while to apply all the security updates to the ISO. I set ssh_timeout to 60m and it normally takes around 20-30 mins to finish with updates - obviously that depends a lot on the speed of the disks you have on your Proxmox server(s) and your Internet connection so YMMV. Similarly ssh_handshake_attempts should be a reasonably big number (I use 500).

Here is the user-data I use for my base OS Packer build (which I use subsequently to build other images using the proxmox-clone builder):

#cloud-config
autoinstall:
  version: 1
  locale: en_GB
  keyboard:
    layout: en
    variant: uk
  identity:
    hostname: ubuntu-server
    username: ubuntu
    password: "$6$FhcddHFVZ7ABA4Gi$9ME1/XEiFHYx8Qh01w6CPqZZE7EDSf2tOc9Ugs89beYrUMyCyCxXzyBovoRwjN/6ipRnxCKeG/3PmJb1zvMAp/"
  manage_etc_hosts: false
  ssh:
    allow-pw: true
    authorized-keys:
      - "ssh-rsa foobarbaz"
    install-server: true
  packages:
    - apt-transport-https
    - ca-certificates
    - bc
    - curl
    - git
    - jq
    - less
    - libnss-myhostname
    - nfs-common
    - nmap
    - openssh-server
    - qemu-guest-agent
    - software-properties-common
    - sshpass
    - wget
    - whois
    - zip
  package_update: true
  package_upgrade: true
  package_reboot_if_required: true
  late-commands:
    - curtin in-target --target=/target -- systemctl start qemu-guest-agent
    - curtin in-target --target=/target -- systemctl enable qemu-guest-agent
  rumcmd:
    - ....

Here's part of the HCL that I use for creating a Proxmox template for a base OS:

source "proxmox-iso" "ubuntu-base-os-cloud-init-template" {
  boot                    = "order=scsi0;ide2"
  boot_command            = ["<esc><wait><esc><wait><f6><wait><esc><wait>", "<bs><bs><bs><bs><bs>", "autoinstall ds=nocloud-net;s=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ ", "-- <enter>"]
  boot_wait               = "${var.boot_wait}"
  cloud_init              = true
  cloud_init_storage_pool = "${var.storage_pool}"
  communicator            = "ssh"
  cores                   = 1
  cpu_type                = "host"
  disks {
    disk_size         = "${var.home_volume_size}"
    format            = "raw"
    storage_pool      = "${var.storage_pool}"
    storage_pool_type = "lvm-thin"
    type              = "scsi"
  }
  http_directory           = "${var.http_directory}"
  insecure_skip_tls_verify = true
  iso_file                 = "${local.iso_filepath}"
  iso_checksum             = "${local.iso_checksum}"
  memory                   = 2048
  network_adapters {
    bridge = "vmbr0"
    model  = "virtio"
  }
  node                   = "${var.proxmox_target_node}"
  onboot                 = false
  os                     = "l26"
  password               = "${var.proxmox_server_pwd}"
  proxmox_url            = "https://${var.proxmox_server_hostname}:${var.proxmox_server_port}/api2/json"
  qemu_agent             = true
  scsi_controller        = "virtio-scsi-pci"
  sockets                = 1
  ssh_handshake_attempts = "500"
  ssh_password           = "${var.ssh_password}"
  ssh_pty                = true
  ssh_timeout            = "${var.ssh_timeout}"
  ssh_username           = "${var.ssh_username}"
  task_timeout           = "${var.task_timeout}"
  template_description   = "Ubuntu ${local.os_version_base} base OS CloudInit Proxmox template."
  template_name          = "${var.os_distro}-${local.os_version_base}-base-os-cloudinit-template"
  unmount_iso            = true
  username               = "${var.proxmox_server_user}"
  vm_id                  = "${var.vm_id}"
}

build {

  sources = ["source.proxmox-iso.ubuntu-base-os-cloud-init-template"]

  provisioner "file" {
    destination = "/tmp/"
    source      = "../scripts"
  }

  provisioner "shell" {
    inline         = ["echo '${var.default_user_pwd}' | sudo -S -u ubuntu bash -c '/tmp/scripts/boot-finished.sh ${local.provisioning_log_filepath}'"]
    inline_shebang = "/bin/bash -e"
    remote_folder  = "/tmp/scripts"
    timeout        = "5m0s"
  }

  provisioner "ansible" {
...

HTHs

Fraser.

[rarrais]

I was facing the same issue, what did the trick for me was to specify the http bind address and port for the autoinstall.

[modem7]

provisioner "ansible" { ...

HTHs

Fraser.

Heya @goffinf

Would it be possible for you to share your ansible configs on github or something similar? I'm interested in getting more into it!

And @EsharkyTheGreat here's my setup - https://github.com/modem7/packer-proxmox-template - might help hopefully!

[cindrmon]

I was facing the same issue, what did the trick for me was to specify the http bind address and port for the autoinstall.

yea, this didn't work in my case. i already specified port numbers and everything. it is still stuck.

[Emanuele94]

Hi, i'm facing the same issue with vsphere-iso, have you ever found a way to fix this ?

[herrmarten]

i am running packer in a dockercontainer defined in a docker-compose.yml and was facing this issue. the solution for me was to set the network_mode: "host" in the service definition of my packer-container.

notice packer binds a webserver-port for the unattended installation which needs to be reachable for the vm you're creating. so proxmox and the machine you're running packer on should be in the same Network(lan/vpn), so they can reach eachothers. (your vm might need an eth-device which is properly bridged or routed)

You can configure which ip packer binds to within the source

    http_bind_address = "0.0.0.0"
    http_port_min = 8802
    http_port_max = 8802

    # PACKER Boot Commands (for Ubuntu-Focal)
    boot_command = [
        "<esc><wait><esc><wait>",
        "<f6><wait><esc><wait>",
        "<bs><bs><bs><bs><bs>",
        "autoinstall ds=nocloud-net;s=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ ",
        "--- <enter>"
    ]

[Emanuele94]

In my case the problem was that since proxmox is nested the dhcp wasn’t working so I had to set the static IP through boot command. (Here is the documentation I used for this: https://unix.stackexchange.com/questions/550021/where-is-the-documentation-for-the-ip-variable-in-initramfs-conf#550028)

[Kibukx]

This is what worked for me. I provided the IP for the VM since I was having issues with DHCP and also specified the bind address and port of the host where packer is running. Opened firewall rule for port just for the sakes and it worked. It takes around 15-20 seconds cloud-init stage be patient.

boot_command = [
        "<esc><wait>",
        "e<wait>",
        "<down><down><down><end>",
        "<bs><bs><bs><bs><wait>",
        "ip=${cidrhost("192.168.68.0/24", 69)}::${cidrhost("192.168.68.0/24", 1)}:${cidrnetmask("192.168.68.0/24")}::::${cidrhost("192.168.68.0/24", 1)}<wait>",
        " autoinstall ds=nocloud-net\\;s=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ ---<wait>",
        "<f10><wait>"
    ]
  
  boot                      = "c"
  boot_wait                 = "6s"
  communicator              = "ssh"

  http_directory            = "pkr-ubuntu-jammy-1/http"
  http_bind_address = "192.168.68.133"
  http_port_min = "8667" 
  http_port_max = "8667"