operating-system icon indicating copy to clipboard operation
operating-system copied to clipboard

Published 20 hours ago • verified publisher icon home-assistant

HASS OS OVA on libvirt+kvm: Reboot after HASS OS update causes crash

Open philpem opened this issue 9 months ago • 4 comments

Describe the issue you are experiencing

I installed the HASS OS 14.2 update this evening, and after the update concluded, I found that my virtual machine was crashed/hung with KVM reporting 100% CPU usage.

The last messages on the virtual console were:

[  OK  ] Finished System Reboot.
[  OK  ] Reached target System Reboot.
reboot: Restarting system

I found this Community post which reports a similar (possibly the same) bug going back to October 2023: https://community.home-assistant.io/t/ha-hangs-and-no-response-after-os-updates-on-kvm/632083/4

I can't put a number on how long it's been happening for me, except that it's been over a year.

Oddly if I go in to the Home Assistant web interface and reboot from there (System -> power menu at top right -> more -> Reboot) then the system doesn't crash. It only seems to happen after a HASS OS update.

What operating system image do you use?

ova (for Virtual Machines)

What version of Home Assistant Operating System is installed?

14.2

Did the problem occur after upgrading the Operating System?

Yes

Hardware details

QEMU/KVM on Libvirt.

VM config:

<domain type="kvm">
  <name>home-assistant</name>
  <uuid>REDACTED</uuid>
  <title>Home Assistant</title>
  <memory unit="KiB">2621440</memory>
  <currentMemory unit="KiB">2621440</currentMemory>
  <vcpu placement="static">2</vcpu>
  <os>
    <type arch="x86_64" machine="pc-i440fx-jammy">hvm</type>
    <loader readonly="yes" type="rom">/usr/share/OVMF/OVMF_CODE.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/home-assistant_VARS.fd</nvram>
    <boot dev="hd"/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <vmport state="off"/>
  </features>
  <cpu mode="host-passthrough" check="none" migratable="on"/>
  <clock offset="utc">
    <timer name="rtc" tickpolicy="catchup"/>
    <timer name="pit" tickpolicy="delay"/>
    <timer name="hpet" present="no"/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled="no"/>
    <suspend-to-disk enabled="no"/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2"/>
      <source file="/mnt/zfs/vmdisks/hassos_ova-5.12.qcow2"/>
      <target dev="hda" bus="ide"/>
      <address type="drive" controller="0" bus="0" target="0" unit="0"/>
    </disk>
    <controller type="usb" index="0" model="ich9-ehci1">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x7"/>
    </controller>
    <controller type="usb" index="0" model="ich9-uhci1">
      <master startport="0"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0" multifunction="on"/>
    </controller>
    <controller type="usb" index="0" model="ich9-uhci2">
      <master startport="2"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x1"/>
    </controller>
    <controller type="usb" index="0" model="ich9-uhci3">
      <master startport="4"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x2"/>
    </controller>
    <controller type="pci" index="0" model="pci-root"/>
    <controller type="ide" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x1"/>
    </controller>
    <controller type="virtio-serial" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x06" function="0x0"/>
    </controller>
    <interface type="bridge">
      <mac address="REDACTED"/>
      <source bridge="bridge-lan"/>
      <model type="e1000"/>
      <link state="up"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x0"/>
    </interface>
    <interface type="bridge">
      <mac address="REDACTED"/>
      <source bridge="bridge-iot"/>
      <model type="e1000"/>
      <link state="up"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
    </interface>
    <serial type="pty">
      <target type="isa-serial" port="0">
        <model name="isa-serial"/>
      </target>
    </serial>
    <console type="pty">
      <target type="serial" port="0"/>
    </console>
    <channel type="spicevmc">
      <target type="virtio" name="com.redhat.spice.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="1"/>
    </channel>
    <input type="tablet" bus="usb">
      <address type="usb" bus="0" port="1"/>
    </input>
    <input type="mouse" bus="ps2"/>
    <input type="keyboard" bus="ps2"/>
    <graphics type="spice" autoport="yes">
      <listen type="address"/>
    </graphics>
    <sound model="ich6">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x04" function="0x0"/>
    </sound>
    <audio id="1" type="spice"/>
    <video>
      <model type="bochs" vram="16384" heads="1" primary="yes"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0"/>
    </video>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="2"/>
    </redirdev>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="3"/>
    </redirdev>
    <memballoon model="virtio">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
    </memballoon>
  </devices>
</domain>

Steps to reproduce the issue

  1. Upgrade from one version of HASS OS to the succeeding version.
  2. Monitor the virtual machine console while HASS OS restarts
  3. Observe HASS OS crashing instead of rebooting the system

Anything in the Supervisor logs that might be useful for us?

Nothing that appears relevant

Anything in the Host logs that might be useful for us?

Nothing that appears relevant

System information

System Information

version core-2025.1.4
installation_type Home Assistant OS
dev false
hassio true
docker true
user root
virtualenv false
python_version 3.13.1
os_name Linux
os_version 6.6.73-haos
arch x86_64
timezone Europe/London
config_dir /config
Home Assistant Community Store
GitHub API ok
GitHub Content ok
GitHub Web ok
HACS Data ok
GitHub API Calls Remaining 5000
Installed Version 2.0.5
Stage running
Available Repositories 1500
Downloaded Repositories 6
Home Assistant Cloud
logged_in false
can_reach_cert_server ok
can_reach_cloud_auth ok
can_reach_cloud ok
Home Assistant Supervisor
host_os Home Assistant OS 14.2
update_channel stable
supervisor_version supervisor-2024.12.3
agent_version 1.6.0
docker_version 27.2.0
disk_total 30.8 GB
disk_used 11.1 GB
healthy true
supported true
host_connectivity true
supervisor_connectivity true
ntp_synchronized true
virtualization kvm
board ova
supervisor_api ok
version_api ok
installed_addons Terminal & SSH (9.16.0), Mosquitto broker (6.5.0), File editor (5.8.0), Network UPS Tools (0.14.0), SQLite Web (4.3.0), ESPHome Device Builder (2024.12.4), eBUSd (24.1.1), Advanced SSH & Web Terminal (20.0.0)
Dashboards
dashboards 5
resources 3
views 21
mode storage
Recorder
oldest_recorder_run 7 January 2025 at 19:20
current_recorder_run 29 January 2025 at 23:41
estimated_db_size 397.23 MiB
database_engine sqlite
database_version 3.47.1
Spotify
api_endpoint_reachable ok

Additional information

No response

philpem avatar Jan 29 '25 23:01 philpem