open-vm-tools icon indicating copy to clipboard operation
open-vm-tools copied to clipboard
verified publisher icon vmware

vmtools unable to start (inactive) when started in AWS ami

Open danilo-patrucco opened this issue 2 years ago • 3 comments

Describe the bug

The problem shows when creating an AMI via Packer in AWS.

The following commands are executed:

yum install -y open-vm-tools
systemctl disable vmtoolsd
rm /usr/lib/systemd/system/vmtoolsd.service
{
  echo "[Unit]"
  echo "Description=Service for virtual machines hosted on VMware"
  echo "Documentation=http://github.com/vmware/open-vm-tools"
  echo "Requires=vgauthd.service"
  echo "#ConditionVirtualization=vmware"
  echo "After=vgauthd.service"
  echo "DefaultDependencies=no"
  echo "Before=cloud-init-local.service"
  echo "[Service]"
  echo "ExecStart=/usr/bin/vmtoolsd"
  echo "TimeoutStopSec=5"
  echo "Type=simple"
  echo "[Install]"
  echo "WantedBy=multi-user.target"
  echo "Also=vgauthd.service"
} > /usr/lib/systemd/system/vmtoolsd.service
ls -ltr /usr/lib/systemd/system/vmtoolsd.service

systemctl enable vmtoolsd
systemctl daemon-reload
systemctl start vmtoolsd
systemctl status vmtoolsd

The output is the following:

    amazon-ebs: ● vmtoolsd.service - Service for virtual machines hosted on VMware
    amazon-ebs:    Loaded: loaded (/usr/lib/systemd/system/vmtoolsd.service; enabled; vendor preset: enabled)
    amazon-ebs:    Active: inactive (dead) since Mon 2023-01-30 16:19:52 UTC; 12ms ago
    amazon-ebs:      Docs: http://github.com/vmware/open-vm-tools
    amazon-ebs:   Process: 8896 ExecStart=/usr/bin/vmtoolsd (code=exited, status=0/SUCCESS)
    amazon-ebs:  Main PID: 8896 (code=exited, status=0/SUCCESS)

When a machine is created using the exported AMI in Vcenter (In this case Terraform is used) the VMware tool is not able to start with the first boot, the machine needs to be completely booted first and then re-created. After that Vcenter can pick up the tool information and start the guest OS

Reproduction steps

  1. Create an AMI in AWS using packer (User RHEL7)
  2. export the AMI to vcenter using terraform
  3. Create a new VM using terraform (select the exported disks and attach them to a template previously created)
  4. Delete the VM (not the disks)
  5. Re-create the VM ...

Expected behavior

vmtools should start and be running (not inactive) even if the machine is not in Vcenter

Additional context

No response

danilo-patrucco avatar Feb 02 '23 21:02 danilo-patrucco

Hi, Thank you for reporting the problem. This is expected behaviour in AWS ami as open-vm-tools is supported only on VMware hypervisors. Can you please check output of command "lscpu | grep -i hyper"? It should report "Hypervisor Vendor: vmware" for VMTools to run. Also, please provide the VM's .vmx file when in issue state, after exporting the VM to Terraform Vcenter.

rprabhud avatar Feb 06 '23 05:02 rprabhud

here is the content of the vmx, please note that I had to mask some values.

.encoding = "UTF-8"
config.version = "8"
virtualHW.version = "19"
vmci0.present = "TRUE"
floppy0.present = "FALSE"
svga.vramSize = "8388608"
numvcpus = "8"
memSize = "8192"
tools.upgrade.policy = "upgradeAtPowerCycle"
sched.cpu.units = "mhz"
vm.createDate = "1675106214069435"
scsi0.virtualDev = "pvscsi"
scsi0.present = "TRUE"
ethernet0.virtualDev = "vmnetxxx"
ethernet0.dvs.switchId = "xx xx xx xx xx xx xx xx-xx xx xx xx xx xx xx xx"
ethernet0.dvs.portId = "2525"
ethernet0.dvs.portgroupId = "dvportgroup-2238"
ethernet0.dvs.connectionId = "244261189"
ethernet0.shares = "normal"
ethernet0.addressType = "vpx"
ethernet0.generatedAddress = "xx:xx:xx:xx:xx:xx"
ethernet0.uptCompatibility = "TRUE"
ethernet0.present = "TRUE"
displayName = "test_machine_vm_01"
guestOS = "rhel7-64"
logging = "FALSE"
toolScripts.afterPowerOn = "TRUE"
toolScripts.afterResume = "TRUE"
toolScripts.beforeSuspend = "TRUE"
toolScripts.beforePowerOff = "TRUE"
tools.syncTime = "FALSE"
uuid.bios = "xx xx xx xx xx xx xx xx-xx xx xx xx xx xx xx xx"
vc.uuid = "xx xx xx xx xx xx xx xx-xx xx xx xx xx xx xx xx"
tools.guest.desktop.autolock = "FALSE"
nvram = "test_machine_vm_01.nvram"
svga.present = "TRUE"
pciBridge0.present = "TRUE"
pciBridge4.present = "TRUE"
pciBridge4.virtualDev = "pcieRootPort"
pciBridge4.functions = "8"
pciBridge5.present = "TRUE"
pciBridge5.virtualDev = "pcieRootPort"
pciBridge5.functions = "8"
pciBridge6.present = "TRUE"
pciBridge6.virtualDev = "pcieRootPort"
pciBridge6.functions = "8"
pciBridge7.present = "TRUE"
pciBridge7.virtualDev = "pcieRootPort"
pciBridge7.functions = "8"
hpet0.present = "TRUE"
sched.cpu.latencySensitivity = "normal"
time.synchronize.allow = "FALSE"
numa.autosize.cookie = "80012"
numa.autosize.vcpu.maxPerVirtualNode = "8"
pciBridge0.pciSlotNumber = "17"
pciBridge4.pciSlotNumber = "21"
pciBridge5.pciSlotNumber = "22"
pciBridge6.pciSlotNumber = "23"
pciBridge7.pciSlotNumber = "24"
scsi0.pciSlotNumber = "160"
ethernet0.pciSlotNumber = "192"
scsi0.sasWWID = "50 05 05 64 35 eb 05 10"
vmotion.checkpointFBSize = "8388608"
vmotion.checkpointSVGAPrimarySize = "8388608"
vmotion.svga.mobMaxSize = "8388608"
vmotion.svga.graphicsMemoryKB = "8192"
monitor.phys_bits_used = "45"
softPowerOff = "FALSE"
svga.guestBackedPrimaryAware = "TRUE"
toolsInstallManager.updateCounter = "4"
toolsInstallManager.lastInstallError = "21004"
guestInfo.detailed.data = "bitness='64' distroName='RedHatEnterpriseServer' distroVersion='7.9' familyName='Linux' kernelVersion='xxxxxxxxxxxxxxx' prettyName='Red Hat Enterprise Linux Server release'"
migrate.hostLog = "test_machine_vm_01-06d8eadc.hlog"
sched.cpu.min = "0"
sched.cpu.shares = "normal"
sched.mem.min = "0"
sched.mem.minSize = "0"
sched.mem.shares = "normal"
migrate.encryptionMode = "opportunistic"
ftcpt.ftEncryptionMode = "ftEncryptionOpportunistic"
viv.moid = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
scsi0:0.deviceType = "scsi-hardDisk"
scsi0:0.fileName = "/vmfs/volumes/634dae89-2adc1263-a0f7-70106f57b010/test/export-ami-0491fdbd34a594a31-dev-sda1.vmdk"
sched.scsi0:0.shares = "normal"
sched.scsi0:0.throughputCap = "off"
scsi0:0.present = "TRUE"
scsi0:1.deviceType = "scsi-hardDisk"
scsi0:1.fileName = "/vmfs/volumes/xxxxxxxx-xxxxxxxx-xxxx-xxxxxxxxxxx/test/export-ami-0491fdbd34a594a31-dev-sdb.vmdk"
sched.scsi0:1.shares = "normal"
sched.scsi0:1.throughputCap = "off"
scsi0:1.present = "TRUE"
sched.swap.derivedName = "/vmfs/volumes/xxxxxxxx-xxxxxxxx-xxxx-xxxxxxxxxxx/test_machine_vm_01_3/test_machine_vm_01-21113906.vswp"
uuid.location = "xx xx xx xx xx xx xx xx-xx xx xx xx xx xx xx xx"
scsi0:1.redo = ""
scsi0:0.redo = ""
vmci0.id = "731191603"
cleanShutdown = "TRUE"
ctkEnabled = "TRUE"
scsi0:0.ctkEnabled = "TRUE"
scsi0:1.ctkEnabled = "TRUE"
scsi0:0.cvctkSize = "80530636800"
scsi0:1.cvctkSize = "161061273600"

This is the oputput of the lscpu command

    amazon-ebs: Hypervisor vendor:     KVM
    amazon-ebs: Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid eagerfpu pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch topoext rsb_ctxsw vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr arat npt nrip_save

There is no way to avoid the hypervisor check ?

danilo-patrucco avatar Feb 06 '23 14:02 danilo-patrucco

No. vmtoolsd service itself is built to run in VMware virtual machines only.

rprabhud avatar Feb 06 '23 16:02 rprabhud