Zigbee extremely slow since last HA update

[Tronnic]

The problem

Hello everyone, since the last HA update my zigbee network doesn’t work anymore. Devices take minutes to hours to receive the signal. Sometimes they don’t work at all. Home assistant turned into a nightmare tonight because all the tries of switching random switches happend hours after going to bed.

I have a raspberry pi 4 with 4gb RAM and a Sonoff Zigbee USB3 Stick.

Home Assistant Core 2024.1.6 Supervisor 2024.01.1 Operating System 11.5 Frontend 20240104.0

Some more Info: I have 26 Zigbee devices, 13 Tuya Smart plugs, 3 Tuya temps sensors, 2 zigbee extenders, some power sensors and door contacts

Here are 2 logs after updating to 2024.1.6 https://drive.google.com/file/d/1fEyUehrZtWYmDT46LuqFfx1XQrT_5WYp/view https://drive.google.com/file/d/1fBsXXh05yFATl1aC5JfZI-5UIAhE0l-Q/view

Someone suggested to update to the latest beta. I am now after updating on version 2024.2.0b8 and still facing the same issues. I think the Zigbee Integration keeps crashing since it shows "initializing" from time to time.

I started debug logging on Zigbee Home Automation and created a massive 50MB log within 3 minutes with over 100.000 entries. Something definitely isn't right. I attached the log below (created after updating to beta). https://drive.google.com/file/d/1pgNxCQa8dRFgE7nkg1UWbsMAKTyt1Scj/view?usp=sharing

What version of Home Assistant Core has the issue?

core-2024.1.6

What was the last working version of Home Assistant Core?

core-2024.1.5

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Zigbee Home Automation

Link to integration documentation on our website

https://www.home-assistant.io/integrations/zha/

Diagnostics information

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[home-assistant[bot]]

Hey there @dmulcahey, @adminiuga, @puddly, @thejulianjes, mind taking a look at this issue as it has been labeled with an integration (zha) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of zha can trigger bot actions by commenting:

• @home-assistant close Closes the issue.
• @home-assistant rename Awesome new title Renames the issue.
• @home-assistant reopen Reopen the issue.
• @home-assistant unassign zha Removes the current integration label and assignees on the issue, add the integration domain after the command.
• @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
• @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

---

zha documentation zha source _{^{(message by IssueLinks)}}

[TheJulianJES]

Please check again with 2024.2.0b9. It'll be out in about an hour.

[puddly]

Please try b9 once it is out, I think it should address some of your issues.

[Tronnic]

Thank you, I will report back once I update tomorrow morning. Good night :)

[chrizstone]

I have the same issue. How can I upgrade to this Pre-Release?

[Tronnic]

error_log-2.txt Unfortunately the update did not help @TheJulianJES @puddly

My system is still acting exactly the same. :/ Even downgrading from a full backup to the working version doesn't help.

I enabled debug logging on b9 for a short period. Here's my log.

[Tronnic]

Just updated to b10, still the same issues. error_log-3.txt

[Tronnic]

Is there anything else I can do to trouble shoot? Would it help to uninstall the zigbee integration and start from scratch? Would another zigbee stick help? I'm desperate and willing to do anything to make my system running again.

[puddly]

From your log, it looks like half of your Tuya plugs are going haywire and are sending about 30 requests per second back to the coordinator. The coordinator is trying to reply to all of them, but they are just sending them way too fast and the requests start queuing up. It is almost as if the plugs cannot actually hear that the coordinator is responding to them.

I have some questions:

1. Can you upload diagnostics JSON for a few of your plugs? I would like to purchase a couple to see if I can replicate this issue. If you have a link to the store where you bought them, that would be ideal.
2. Did you modify the configuration for the plugs in any way? E.g. with ZHA toolkit.
3. Can you try rebooting all of your plugs to reset them? After you do that, reload the ZHA Integration.

[Tronnic]

These are the exact plugs I bought: https://a.aliexpress.com/_EH1dzch Tuya Zigbee Plug V2 You might also find them on Amazon: https://amzn.eu/d/9jO30rt Where do you live? I have an unused one, I'll gladly send it to you as a thanks for your great work. I'm from Germany so if you're inside the EU this could work out :)

Here's the diagnostic of 3 plugs: zha-27b9445dd724bb27a0c757c6c11bc5bc-_TZ3000_w0qqde0g TS011F-bfcb4ad69f8f51db82dc70b00305b5f3.json zha-27b9445dd724bb27a0c757c6c11bc5bc-_TZ3000_w0qqde0g TS011F-c149586c2e9ac7551ebeebf93556977b.json zha-27b9445dd724bb27a0c757c6c11bc5bc-_TZ3000_w0qqde0g TS011F-5f8f0525dc7774c771e9c5c2673a9ea4.json

I did not modify any configuration trough zha toolkit. I'm pretty new to HA. Have been using it since about 6-8 weeks and switched over my Smarthome from Alexa to HA. So not that super experienced yet.

I will try unplugging and replugging all plugs when I'm at home. Still at work. About 5 hrs until I can try.

Thank you a lot for helping!

[chrizstone]

I have the same Tuya plugs and I am also now trying to unplug them and see if it’s getting better. I am just wondering why my Tuya Plugs worked before the HA update…

[dmulcahey]

I have the same Tuya plugs and I am also now trying to unplug them and see if it’s getting better. I am just wondering why my Tuya Plugs worked before the HA update…

The answer to that question is in here: https://github.com/home-assistant/core/issues/107200#issuecomment-1932490512

TL;DR we fixed a bug that was causing other issues and exposed this problem in the process.

[chrizstone]

Ok, understood. But I am not using ZHA. using Z2M… But I guess it’s the same issue then.

I can already confirm, since I unplugged Tuya devices everything is back to normal

[une18]

error_log-2.txt Unfortunately the update did not help @TheJulianJES @puddly

My system is still acting exactly the same. :/ Even downgrading from a full backup to the working version doesn't help.

I enabled debug logging on b9 for a short period. Here's my log.

Had the same issue. Rolled back to 24.1.5 and this helped. Tried to upgrade all, and system broke again. Rolled back to 24.1.5 and all OK.

I have Athom Zigbee bridge, Ikea Tradfri light and switch, several switches, leak sensors etc. Most stuff tuya.

Wireless switches worked, device Logbook registered button presses immediately, but all automations were down. When I clicked in HA Run automation, light turned on. When I clicked physical button, nothing happened (press was registered in Logbook). Yesterday I had the feeling that the automation and the device have different ON/OFF commands (like a typho somewhere). Created new automation, but it did not work.

My zigee setup seemed bulletproof umtil now..., just got myself 5 Moes thermostats, that I tested, but did not have time to install yet.

[jandirk39]

Hi,

It doesn’t have to be related, but this sounds just like I had a couple of weeks ago. The issue for me was with port forwarding in my router. I had 8080 to 8123 so I could reach ha from outside. It became verrrrryyyyy slow as well. I deleted the port forwarding and my commands would trigger devices immediately again. Now for some strange reason I can forward again without issues

[HvanWijlen]

My problems with ZHA and specially automations started after version HAOS 2023.12.4. Haven't been able to use any updates after that without my tuya light switches automations start being incredibly slow or automations not working at all. According release notes there was a change in automation to unified actions instead of service calls. Since then my problems started.

Today tested again with lastest release 2024.2.1 but still immediatly my switches stop synching by automations. Same result as multiple previous releases 2024.1.x. Reverted back to latest release 2023 and working again.

Not sure if related in anyway or specific to my situation but thought perhaps this information is helpfull.

In proces of getting an ZigBee MQTT lan gateway to move some swithes from ZHA to MQTT in the hope this will help in my situation.

[homeworkhonc]

I have the same issue as OP - just updatet today and wife started complaining :) 1 Minute after restart the devices are getting slow, 1 minute further in and everything stops working. are you guys still on it? can i do something else or am i forced to roll back?

[dmulcahey]

I have the same issue as OP - just updatet today and wife started complaining :) 1 Minute after restart the devices are getting slow, 1 minute further in and everything stops working. are you guys still on it? can i do something else or am i forced to roll back?

What coordinator do you have?

[une18]

I have the same issue as OP - just updatet today and wife started complaining :) 1 Minute after restart the devices are getting slow, 1 minute further in and everything stops working. are you guys still on it? can i do something else or am i forced to roll back?

Roll back, as there is no progress for over 1 week.

[homeworkhonc]

@dmulcahey i am also using the SONOFF Zigbee 3.0 USB and have a 6 of Tuya power plugs...

[dmulcahey]

@dmulcahey i am also using the SONOFF Zigbee 3.0 USB and have a 6 of Tuya power plugs...

If you have source routing on (yaml config) try disabling it

[homeworkhonc]

no but i found that i have a Tuya config im my quirks https://github.com/zigpy/zha-device-handlers/blob/dev/zhaquirks/tuya/init.py could that be related?

[homeworkhonc]

@dmulcahey Hi, im not aware that i have source routing acitve in my configuration.yaml - am i looking in the wrong place?

Also Rollback didn't work - had a full backup from mid January but it didnt work. i can't comprehend why - it seems so counterintuitive. Any ideas what i can do?

[chrizstone]

Unplug the Tuya devices was my only solution so far…

[Ditskys]

having same issue, found disabling tuya helped but found that its like HA is delayed every so many seconds. am hunting through logs to find whats causing it to hang

[homeworkhonc]

something very strange happend: I didn't change anything - didnt unplug anything, just let it sit in this non functionable way.... but suddenly yesterday evening everything worked again. What kind of vodoo is this?

[une18]

Accidentally installed HA upgrade again. For 10 minutes all seemed ok, but had to recover, to get Tuya switches and automations working again in the evening. Any fix on the way for this? This is not a single person issue. I have not seen a solution other than "unplug Tuya devices". All is working before the last upgrade...

[rtenklooster]

Same here. I have my zigbee network running for several years. Never had those kind of issues. Running 2024.2.4 Things I notice: Zigbee is very slow: turning lights on/ off can take 5-10 seconds. Zigbee groups are not responding a lot of times. Devices are going offline.

I use Z2M with the Sonoff -P USB stick, in Z2M debug logs I see (more then 10 messages per second) from some of my tuya plugs. I have the feeling those plugs are causing some kind of zigbee-ddos attack, causing cognestion.

I disconnected 3 plugs and everything feels much more responsive.

Example Z2M logbook entries for a plug that's still connected: Debug 2024-02-28 11:40:18Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:18Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:18Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:18Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:19Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:19Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:19Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:19Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:19Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:20Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:20Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Warning 2024-02-28 11:40:22

[une18]

Same here. I have my zigbee network running for several years. Never had those kind of issues. Running 2024.2.4 Things I notice: Zigbee is very slow: turning lights on/ off can take 5-10 seconds. Zigbee groups are not responding a lot of times. Devices are going offline.

I use Z2M with the Sonoff -P USB stick, in Z2M debug logs I see (more then 10 messages per second) from some of my tuya plugs. I have the feeling those plugs are causing some kind of zigbee-ddos attack, causing cognestion.

I disconnected 3 plugs and everything feels much more responsive.

Example Z2M logbook entries for a plug that's still connected: Debug 2024-02-28 11:40:18Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:18Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:18Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:18Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:19Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:19Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:19Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:19Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:19Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:20Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Debug 2024-02-28 11:40:20Received Zigbee message from 'CV Ketel', type 'attributeReport', cluster 'genBasic', data '{"65506":31,"65508":0,"appVersion":69}' from endpoint 1 with groupID 0 Warning 2024-02-28 11:40:22

Might it be that those Tuya devices are waitnig for some confirmation that the message was received. In case of no confirmation another message is sent...

[HvanWijlen]

Still same experience but worked around it. Before all working flawless after updates is just unworkable and had to restored to earlier version. Now was able to split my zigbee network in two by using half on zha and other half on z2m. Things have improved a lot and is workable.

Do notice extra messages from Tuya device.

Also having some devices flashed with openbeken firmware. Those give a constant flow now of messages about power consumption (unlike before). Also a configured (sum) helper shows constant change messages, probable related to those device updates send.

So most likely my slowness comes from excessive messaging on the network that somehow is related to changes in the updates and devices used. Only can state again before wasnt like that. Difference is like day and night between earlier (2023) and current versions. Having a workaround, I am and will stay on the new version.