zigbee2mqtt icon indicating copy to clipboard operation
zigbee2mqtt copied to clipboard

Zigbee devices became unavailable in HA after restart or reboot of HA

Open Motya-HA opened this issue 1 year ago β€’ 59 comments

What happened?

After recent update to laters version Z2M (1.33.0-1) all Hue Zigbee devices are droping from network and become unavailable. Requires Z2M restart to bring them back

What did you expect to happen?

HUE zigbee devices will not drop from network and be stable

How to reproduce it (minimal and precise)

No response

Zigbee2MQTT version

1.33.0-1

Adapter firmware version

6.10.3.0 build 297

Adapter

ITEAD_SONOFF_Zigbee_3.0_USB_Dongle_Plus_V2

Debug log

No response

Motya-HA avatar Sep 04 '23 15:09 Motya-HA

Same here. I don't have any error message in the logs. It works well, then one random command fails and the device is not connected anymore to the network.

I have a Conbee II adapter.

A reboot of the RPi solves the issue until next failure.

Can it be an incompatibility between some new frames in the last update and the firmware of the adapter?

Bcow7 avatar Sep 04 '23 22:09 Bcow7

Correction. These are not just Hue devices. Recent check showed that pretty much all Zigbee devices with exception to inovelli dimmers Screenshot 2023-09-04 at 11 12 17 PM

Motya-HA avatar Sep 05 '23 03:09 Motya-HA

Just a comment based on my setup. I had problems with that firmware on the ZBDongle-E, I'm now on 7.1.1 and it is working much better.

KoalaWerewolf avatar Sep 05 '23 07:09 KoalaWerewolf

@KoalaWerewolf Thanks for suggestion. I can't see 7.1.1 firmware in that site: https://github.com/xsp1989/zigbeeFirmware/tree/master/firmware/Zigbee3.0_Dongle-NoSigned/EZSP

Motya-HA avatar Sep 05 '23 12:09 Motya-HA

I used this way of flashing: https://darkxst.github.io/silabs-firmware-builder/ and the firmware from here: https://cloud.time-vortex.net/s/Jy2x6tj2GNx2QZn (check this post and the thread above: https://github.com/Koenkk/zigbee-herdsman/issues/319#issuecomment-1634886994)

KoalaWerewolf avatar Sep 05 '23 13:09 KoalaWerewolf

@KoalaWerewolf Thanks I'll try when I'll be back home (traveling now) Can I do it with Mac or Linux? I do not have any Windows machine at home

Thanks

Motya-HA avatar Sep 05 '23 13:09 Motya-HA

I’m sure you can, but I don’t know if that webflasher works.

KoalaWerewolf avatar Sep 05 '23 13:09 KoalaWerewolf

Same here. Almost all ZigBee devices are disonnecting. It's a mess. I'm on the Sonoff ZigBee 3.0 USB dongle with latest custom firmware.

Grandma-Betty avatar Sep 05 '23 15:09 Grandma-Betty

Some more details - every time there is an update or home assistant restart, all Z2M devices become disconnected. Restart of Z2M add on fixing the issue. Any plans to address this bug?

Motya-HA avatar Sep 09 '23 09:09 Motya-HA

I have this issue as well: my entities provided by Zigbee2MQTT (via Mosquitto broker) become "Unavailable" when I restart Home Assistant, until I also restart the Z2M add-on. I'm not sure how to debug this

stevehollaar avatar Sep 09 '23 21:09 stevehollaar

Are they disconnected from Zigbee2MQTT/the ZigBee network or just "unavailable" in HA? The later would most likely be an issue with retained MQTT topics (or the lack thereof), the former would be more serious.

mundschenk-at avatar Sep 11 '23 21:09 mundschenk-at

They became "unavailable" in HA. In order to make them "available" I need to restart Z2M add-on. sometimes several times. I've updated the topic subject

Motya-HA avatar Sep 11 '23 22:09 Motya-HA

This debugging thread on the MQTT side of things may be helpful: https://github.com/home-assistant/core/issues/99936#issuecomment-1716028130 It seems Z2M may be dropping entity names sent to the config topic, leading to a conflict when it falls back to the device class. There's a bugfix in the upcoming 2023.9.2 release that may help, but I'm unsure if a fix is also needed in Z2M

stevehollaar avatar Sep 12 '23 23:09 stevehollaar

I think fix is needed in Z2M in a first place. All these issues have started after recent update for Z2M

Motya-HA avatar Sep 13 '23 00:09 Motya-HA

Same problem. Devices and groups are unavailable after HA restart until I manualy restart Z2M. 3 Z2M networks: Sonoff (Z2M docker 1.33.1) Sonoff (Z2M docker 1.33.1-dev) zzh! (Z2M HA addon 1.33.0)

rossowl avatar Oct 05 '23 07:10 rossowl

@rossowl do you still encounter that as I had to downgrade to 2023.8.4 HA due to UI slow/laggy so want to confirm if I'll have to deal with that now - every HA restart == Z2M restart?

cpuks avatar Oct 22 '23 15:10 cpuks

I confirm I still have this issue: Z2M HA addon 1.33.1-1 Home Assistant 2023.10.4 Supervisor 2023.10.0 Operating System 11.0 Frontend 20231005.0

Motya-HA avatar Oct 22 '23 16:10 Motya-HA

OK I didn't have that when running external mqtt/z2m now with fresh HA VM I've got mqtt broker from addons and it's an issue.

cpuks avatar Oct 22 '23 16:10 cpuks

now it requires 2 or even 3 restarts of Z2M add-on for some devices to become responsive :-(

Motya-HA avatar Oct 22 '23 21:10 Motya-HA

OK I didn't have that when running external mqtt/z2m now with fresh HA VM I've got mqtt broker from addons and it's an issue.

The mqtt add-on is very buggy at the moment

francisp2 avatar Oct 23 '23 14:10 francisp2

The mqtt add-on is very buggy at the moment

I do not have any issues with Z-Wave JS UI (uses MQTT as well)

Motya-HA avatar Oct 23 '23 15:10 Motya-HA

Z-Wave JS UI does not use MQTT when integrated with HA.

mundschenk-at avatar Oct 23 '23 15:10 mundschenk-at

"The Z-Wave JS UI add-on provides an additional control panel, allowing you to configure every aspect of your Z-Wave network. It provides a decoupled gateway which can communicate using Z-Wave JS WebSockets (used by the Home Assistant Z-Wave JS integration) and MQTT (even simultaneously)"

Motya-HA avatar Oct 23 '23 16:10 Motya-HA

Yes, you can enable MQTT (disabled by default!). But the Z-Wave JS integration does not use it even if you do. So unless you have enabled the MQTT client in the Z-Wave JS UI settings and don't use the Z-Wave JS integration, your HA instance does not use MQTT to access your Z-Wave devices.

mundschenk-at avatar Oct 23 '23 16:10 mundschenk-at

I did enable it

Motya-HA avatar Oct 23 '23 16:10 Motya-HA

I can confirm that with external mqtt everything works. Intel NUC: Hass OS MQTT addon disabled

RPi4 with docker: eclipse-mosquitto 2x zigbee2mqtt

rossowl avatar Nov 02 '23 12:11 rossowl

Installed Zigbee2MQTT in Linux box (stoped add-on in HA). Rebuild whole Zigbee network. Same issue. After restart of HA (Zigbee2MQTT run on Linux box interrupted) all my Zigbee devices becaem unavailable. Need 2, 3 reboot of linux box with Zigbee2Mqtt to make all devices responsive again

Motya-HA avatar Nov 11 '23 06:11 Motya-HA

Same for me, restart of ha or the mqtt server are causing zigbee2mqtt completely offline. As I can see:

After a restart of the mqtt server, the entries in the homerassistant mqtt folder are not rebuild. So they don't have a retain flag and are not created --> device offline.

Fixing the problem of the homeassistant folder is simple: set a retain flag here

Also some values in the zigbee2mqtt folder are also not updated, because they get only updated on device response. A solution could be to update those values from time to time, or to handle reconnects from mqtt server better.

A reconnect in the mqtt library should create a callback that sends out all values again.

I think this should fix all problems we have.

PS: I saw an option for setting retain flag in the settings, but I want to see if there are connection errors, so setting the retain flag for all values could be a problem?

merlink01 avatar Dec 05 '23 08:12 merlink01

@merlink01

Fixing the problem of the homeassistant folder is simple: set a retain flag here

Can you please explain how?

Motya-HA avatar Dec 05 '23 19:12 Motya-HA

I don't know what mqtt library is used here, in paho on python a publish function looks like this:

publish(topic, payload=None, qos=0, retain=False)

Topic is the path in the mqtt tree structure Payload is the json containing the information where homeassistant searches for the values qos is the adjustment how the messages are delivered the higher the more reliable, but causes more traffic retain: tell the mqtt server to send the last good value to new clients or in case of connection loss to the same client again

So I think in this case it could help.

merlink01 avatar Dec 05 '23 20:12 merlink01