Timeout waiting for SSH

[ssplatt]

Overview of the Issue

Attempting to use VMWare desktop 17 in a linux ci/cd environment (headless) results in an ssh timeout

Reproduction Steps

use packer with vmware workstation 17 on Ubuntu in Github Actions.

Plugin and Packer version

1.11.2

Simplified Packer Buildfile

packer {
  required_plugins {
    vmware = {
      version = "~> 1"
      source = "github.com/hashicorp/vmware"
    }
  }
}

variable "vmx_path" { default = "/home/runner/.vagrant.d/boxes/randomvagrantbox/version/vmware_desktop/box.vmx" }

source "vmware-vmx" "box" {
  source_path       = "${var.vmx_path}"
  output_directory  = "./vmware_desktop/"
  ssh_username      = "vagrant"
  ssh_password      = "vagrant"
  headless          = "true"
}

build {
  sources = ["source.vmware-vmx.box"]
}

Operating system and Environment details

github actions, ubuntu.

Log Fragments and crash.log files

2024/08/15 20:44:25 packer-plugin-vmware_v1.1.0_x5.0_linux_amd64 plugin: 2024/08/15 20:44:25 A network mapper configuration file does not exist in the default path: /etc/vmware/netmap.conf
2024/08/15 20:44:25 packer-plugin-vmware_v1.1.0_x5.0_linux_amd64 plugin: 2024/08/15 20:44:25 Checking alternate path for network mapper configuration file: /etc/vmware/networking
2024/08/15 20:44:25 packer-plugin-vmware_v1.1.0_x5.0_linux_amd64 plugin: 2024/08/15 20:44:25 Located the network mapper configuration file: /etc/vmware/networking
2024/08/15 20:44:25 packer-plugin-vmware_v1.1.0_x5.0_linux_amd64 plugin: 2024/08/15 20:44:25 GuestIP discovered device matching nat: vmnet8
2024/08/15 20:44:25 packer-plugin-vmware_v1.1.0_x5.0_linux_amd64 plugin: 2024/08/15 20:44:25 Lookup up IP information...
2024/08/15 20:44:25 packer-plugin-vmware_v1.1.0_x5.0_linux_amd64 plugin: 2024/08/15 20:44:25 IP lookup failed: unable to determine MAC address
2024/08/15 20:44:25 packer-plugin-vmware_v1.1.0_x5.0_linux_amd64 plugin: 2024/08/15 20:44:25 [DEBUG] Error getting SSH address: IP lookup failed: unable to determine MAC address
2024/08/15 20:44:30 packer-plugin-vmware_v1.1.0_x5.0_linux_amd64 plugin: 2024/08/15 20:44:30 [DEBUG] SSH wait cancelled. Exiting loop.
==> vmware-vmx.box: Timeout waiting for SSH.
==> vmware-vmx.box: Timeout waiting for SSH.

more info in this thread as well https://github.com/hashicorp/packer-plugin-vmware/issues/229

[ssplatt]

fwiw vmware workstation 17.6.0 came out and the issue is still present. I also tried using the ubuntu-20.04 runner and the issue is still present.

[tenthirtyam]

Which guest operating system is the image?

[tenthirtyam]

This issue has not received any update and will be closed in 7 days if it does not receive any activity by then.

Issues on GitHub are intended to be related to bugs or feature requests, so we recommend using our other community resources instead of asking here if you have a question.

• HashiCorp Discus
• HashiCorp Packer Guides
• HashiCorp Packer Community

[ssplatt]

centos 9 stream

[tenthirtyam]

Can you provide the repository for reproduction?

[ssplatt]

https://github.com/ssplatt/vagrant-box-centos-stream-9

[tenthirtyam]

Based on the logs in the GitHub Actions doesn't look like the virtual machine is actually successfully booting in Workstation.

I'd suggest setting this up to use an on-premises hosted runner where you can monitor progress - perhaps disabling headless for on-premises to see how far it actually gets in the build process.

I'm not sure if a the Github runners will actually allow the machine to start - never tired that.

[tenthirtyam]

Based on what I read VMware Workstation cannot run on GitHub-hosted runners. You need to use self-hosted runners with VMware Workstation installed. GitHub-hosted runners do not support running virtual machines due to their environment limitations.

[ssplatt]

I have virtualbox builds working 🤷

[tenthirtyam]

Perhaps VirtualBox is not as strict on requirements, for example, support of nested virtualization.

Recommend testing this with a self-hosted runner and then compare. At the moment, this does not appear to be an issue with the plugin.

[Stromweld]

@tenthirtyam github actions as of April this year upgraded the sizing of their base hosted runners to sizes that support nested virtualization now. https://github.com/actions/runner-images/discussions/7191#discussioncomment-9018826

[tenthirtyam]

But will it support VMware Workstation Pro 17.6+?

Also, keep in mind the licensing change for Personal Use and the need to touch the license file if not licensed for commercial use.

[tenthirtyam]

@ssplatt you'll probably also need to update vmware-publish.yml to create the license file since otherwise it's likely to pick up as Player for now.

[Stromweld]

that would have to be scripted in your workflow file to download, install and touch that file to get it to work. individual users private and public repos I'm sure falls under personal use, but for corporate repos they'd need to configure it with license then.

[ssplatt]

i have sudo touch /etc/vmware/license-ws-foo in my script. is there a different command I should be using? Could there be a different reason that it can't determine the mac address?

[tenthirtyam]

In the meantime, for troubleshooting, please test with a self-hosted runner and disable headless.

[tenthirtyam]

I see the problem after reproducing it. The source CentOS box does not have a network interface attached to the VM, thus it doesn't get an address. There's only the loopback lo interface so it will never get an IP address. 😃

[tenthirtyam]

To address the issue for not returning the MAC you'll need to add a network adapter to the virtual machine during the clone.

source "vmware-vmx" "box" {
  source_path       = "${var.vmx_path}"
  output_directory  = "./vmware_desktop/"
  communicator      = "ssh"
  ssh_username      = "vagrant"
  ssh_password      = "vagrant"
  ssh_timeout       = "10m"
  shutdown_command  = "sudo shutdown -h now"
  format            = "vmx"
  headless          = true
  vmx_data = {
    "ethernet0.connectionType" = "nat"
    "ethernet0.addressType" = "generated"
    "ethernet0.virtualDev" = "vmxnet3"
    "ethernet0.linkStatePropagation.enable" = "TRUE"
    "ethernet0.present" = "TRUE"
  }
}

This should get you past your current issue obtaining getting the MAC when running on a GitHub-hosted runner but you may run into additional issues past this expecting the DHCP address on the GitHub-hosted runner.

[tenthirtyam]

Since this issue does appears to be environmental, I'm marking this issue as closed as it does not point to an issue with the plugin itself.

The following is returned from the runner, but I updated the action to use tree to print the contents of /etc/vmware to ensure the files were present.

See https://github.com/tenthirtyam/vagrant-box-centos-stream-9/actions/runs/11338107668/job/31530783035.

Feel free to provide any additional updates or questions.

[ssplatt]

FWIW, I was able to find a couple of commands which helped some in a different environment:

sudo vmware-modconfig --console --install-all
sudo modprobe -a vmw_vmci vmmon

and I was able to confirm that on a remote machine, running packer with x11 forwarded, the build does pop up the license screen, which asks for this on the cli:

==== AUTHENTICATING FOR org.freedesktop.policykit.exec ===

Authentication is needed to run `/usr/lib/vmware/bin/vmware-setup-helper' as the super user

Authenticating as: root

but I cannot find any cli options for vmware-setup-helper to get me past this. So, the licensing is definitely blocking at this time. I have some forum posts up in different places (even at broadcom which is a ghost town, i opened a post there 6 months ago on this topic and have had no responses).

[tenthirtyam]

I have a fix for personal use license tracked and plan to include it in the next release after consolidation of the drivers is completed.

[ssplatt]

another update: packer/vmware does work on Digital Ocean Ubuntu 22.04 servers. Still no luck on Github Actions although it does look like the correct CPU flags are set and accessible. Amazon AWS does not present the correct CPU flags so VMware Workstation Pro will not run there.

I found the setting to disable/skip the first run window:

mkdir -p "$HOME/.vmware"
echo "pref.wspro.firstRunDismissedVersion = \"17.6.1\"" > "$HOME/.vmware/preferences"

and there were several libs that had to be installed:

sudo apt-get install -y software-properties-common wget build-essential libxcb-render0 libpcsclite1 libxcb-shm0 libaio1 libxi6 libxinerama1 libxcursor1 libxtst6

but then packer with vmware workstation runs successfully on Digital Ocean.