core icon indicating copy to clipboard operation
core copied to clipboard

Failed to deliver message: <sl_Status.ZIGBEE_DELIVERY_FAILED: 3074> (not for RF interference)

Open stegitto opened this issue 1 year ago • 35 comments

The problem

Never had an issue and now I'm getting the following error on various Zigbee devices - Failed to deliver message: <sl_Status.ZIGBEE_DELIVERY_FAILED: 3074>

Referring to: https://github.com/home-assistant/core/issues/124516

What version of Home Assistant Core has the issue?

core-2024.9.2

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

ZHA Zigbee

Link to integration documentation on our website

No response

Diagnostics information

home-assistant_zha_2024-09-20T17-37-58.067Z.log

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

Hi All

Tried to downgrade to core-2024.8.2 and core-2024.8.1, same issue. Zigbee coordinator: SONOFF Zigbee 3.0 USB Dongle Plus V2. Coordinator firmware release 7.4.4 ZHA Visualization reports green devices while there is no way to control them. Using channel 25 without RF interference.

Stefano

stegitto avatar Sep 20 '24 21:09 stegitto

Hey there @dmulcahey, @adminiuga, @puddly, @thejulianjes, mind taking a look at this issue as it has been labeled with an integration (zha) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of zha can trigger bot actions by commenting:

  • @home-assistant close Closes the issue.
  • @home-assistant rename Awesome new title Renames the issue.
  • @home-assistant reopen Reopen the issue.
  • @home-assistant unassign zha Removes the current integration label and assignees on the issue, add the integration domain after the command.
  • @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
  • @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

(message by CodeOwnersMention)


zha documentation zha source (message by IssueLinks)

home-assistant[bot] avatar Sep 20 '24 21:09 home-assistant[bot]

Please attach both pieces of diagnostics information, I showed in the screenshot where you can download it: https://github.com/home-assistant/core/issues/124516#issuecomment-2364398602

puddly avatar Sep 20 '24 21:09 puddly

Same issue being experienced across multiple devices. Nothing has been added to or changed in the ZHA config. Same error with an error log of: Logger: homeassistant Source: components/zha/helpers.py:1291 First occurred: 07:14:45 (1 occurrences) Last logged: 07:14:45

Error doing job: Task exception was never retrieved (None) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/zha/zigbee/cluster_handlers/init.py", line 67, in wrap_zigpy_exceptions yield File "/usr/local/lib/python3.12/site-packages/zha/zigbee/cluster_handlers/init.py", line 85, in wrapper return await RETRYABLE_REQUEST_DECORATOR(func)(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/zigpy/util.py", line 136, in retry return await func() ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/zigpy/quirks/init.py", line 254, in command return await self.request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/zigpy/zcl/init.py", line 378, in request return await self._endpoint.request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/zigpy/endpoint.py", line 265, in request return await self.device.request( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/zigpy/device.py", line 339, in request await send_request() File "/usr/local/lib/python3.12/site-packages/zigpy/application.py", line 834, in request await self.send_packet( File "/usr/local/lib/python3.12/site-packages/bellows/zigbee/application.py", line 827, in send_packet raise zigpy.exceptions.DeliveryError( zigpy.exceptions.DeliveryError: Failed to deliver message: <sl_Status.ZIGBEE_DELIVERY_FAILED: 3074>

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/components/zha/helpers.py", line 1289, in handler return await func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/zha/light.py", line 181, in async_turn_on await self.entity_data.entity.async_turn_on( File "/usr/local/lib/python3.12/site-packages/zha/application/platforms/light/init.py", line 413, in async_turn_on result = await self._on_off_cluster_handler.on() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/zha/zigbee/cluster_handlers/init.py", line 84, in wrapper with wrap_zigpy_exceptions(): File "/usr/local/lib/python3.12/contextlib.py", line 158, in exit self.gen.throw(value) File "/usr/local/lib/python3.12/site-packages/zha/zigbee/cluster_handlers/init.py", line 76, in wrap_zigpy_exceptions raise ZHAException(message) from exc zha.exceptions.ZHAException: Failed to send request: Failed to deliver message: <sl_Status.ZIGBEE_DELIVERY_FAILED: 3074>

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/components/script/init.py", line 707, in _async_run return await self.script.async_run(script_vars, context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 1795, in async_run return await asyncio.shield(create_eager_task(run.async_run())) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 463, in async_run await self._async_step(log_exceptions=False) File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 527, in _async_step self._handle_exception( File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 557, in _handle_exception raise exception File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 525, in _async_step await getattr(self, handler)() File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 1074, in _async_if_step await self._async_run_script(if_data["if_then"]) File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 1268, in _async_run_script result = await self._async_run_long_action( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 726, in _async_run_long_action return await long_task ^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 1795, in async_run return await asyncio.shield(create_eager_task(run.async_run())) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 463, in async_run await self._async_step(log_exceptions=False) File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 527, in _async_step self._handle_exception( File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 557, in _handle_exception raise exception File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 525, in _async_step await getattr(self, handler)() File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 763, in _async_call_service_step response_data = await self._async_run_long_action( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 726, in _async_run_long_action return await long_task ^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/core.py", line 2761, in async_call response_data = await coro ^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/core.py", line 2804, in _execute_service return await target(service_call) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 996, in entity_service_call single_response = await _handle_entity_call( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 1068, in _handle_entity_call result = await task ^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/light/init.py", line 626, in async_handle_light_on_service await light.async_turn_on(**filter_turn_on_params(light, params)) File "/usr/src/homeassistant/homeassistant/components/zha/helpers.py", line 1291, in handler raise HomeAssistantError(err) from err homeassistant.exceptions.HomeAssistantError: Failed to send request: Failed to deliver message: <sl_Status.ZIGBEE_DELIVERY_FAILED: 3074>

i8nemo avatar Sep 21 '24 21:09 i8nemo

@i8nemo Please attach both pieces of diagnostics information, I showed in the screenshot where you can download it: https://github.com/home-assistant/core/issues/124516#issuecomment-2364398602

puddly avatar Sep 21 '24 21:09 puddly

This error came back for me today as well. Worked two weeks straight no problem and then received it. No changes to Core or OS.

ecchodun avatar Sep 23 '24 12:09 ecchodun

I encountered this error on a fresh HA OS installation, with a Sonoff USB Zigbee adapter. Devices automatically discovered and added to the network would display in HA, but give the above error when issuing commands.

Removed devices, and re-added, issue no longer appears. Not sure if it's related to that but I figured I'd throw it out there.

AndrewUHD avatar Sep 25 '24 02:09 AndrewUHD

Hi there , is any troubleshooting / analysis action running?

stegitto avatar Sep 26 '24 19:09 stegitto

https://www.reddit.com/r/homeassistant/comments/1fcldu5/sl_statuszigbee_delivery_failed_3074/

stegitto avatar Sep 27 '24 20:09 stegitto

@stegitto i had issues with ikea outlets and the usb repeaters where they lock up and refuse to route traffic. If you truly want a resolution please humor me and remove them from your network for a day or two and let's see if your issues go away. I also noticed in your config entry diagnostics that the EZSP counters seem to show various failures without many successes but that is just from a quick peek on mobile. Personally, I'd start with removing the IKEA devices and see if the issues clear up. I know this isn't something folks like to hear but it is worth trying. It's also good to understand that the only control we have is asking the coordinator to send a message for us. After that it's basically all out of our control. There isn't much we can do to influence things...

dmulcahey avatar Sep 27 '24 21:09 dmulcahey

Hi dmulcahey. Thank you for your message. I will try. I’ve been using the same devices for two years, I cannot figure out what is changed..

stegitto avatar Sep 27 '24 21:09 stegitto

Sometimes the software stack on the routers can just lock up... you can try a less drastic approach first: pull the power from all the ikea devices for like 45s then put it back. Give it a bit and see if things improve.

dmulcahey avatar Sep 27 '24 21:09 dmulcahey

I tried almost everything. Downgrades, resets and reconnects, I’m not able to identify the problem. Many people started struggling during the past month, several of them with ZHA and Zigbee USB dongles. I tried to downgrade HA and to upgrade the coordinator firmware. Unplugging some devices for days and then back on, let the zigbee network run only for some hours. The ZHA visualization map is always green during the “network crash”. Nearest WiFi channel used is 7. No USB3 ports, I use a long extension cable. Moved from Pi4 to Proxmox. No way.

stegitto avatar Sep 27 '24 22:09 stegitto

Dude, if you want help... try what I am suggesting. If you wanna rehash your frustration continuously I can't help you. I get that issues stink. I have a 200 device network working just fine and there are MANY other users without issues. If you look at the analytics there are more than 70k ZHA users: https://analytics.home-assistant.io/integrations/. There are not 70k folks reporting issues. Unfortunately, Zigbee is not a perfect technology and there are individual circumstances that cause individual issues. We do our best to help folks.

dmulcahey avatar Sep 27 '24 22:09 dmulcahey

I thought doing a recap could help. Now I understand I’m too old to keep playing with these toys. Goodbye.

stegitto avatar Sep 27 '24 22:09 stegitto

I have a 200 device network working just fine and there are MANY other users without issues.

Perhaps you could give some guidance on what type of Zigbee devices you're utilizing without error and using what kind of Zigbee Coordinator.

ecchodun avatar Sep 27 '24 22:09 ecchodun

I have a 200 device network working just fine and there are MANY other users without issues.

Perhaps you could give some guidance on what type of Zigbee devices you're utilizing without error and using what kind of Zigbee Coordinator.

Host is a HA Blue

Coordinator is a TubeZb SI MGM24 connected via USB

Devices are a mix of inovelli blue switches, Philips Hue bulbs, centralite sensors and Aqara sensors and ~24 Third Reality energy monitoring outlets.

IMO router devices are the most important consideration after the coordinator. Some devices just don't route well.

I'll also never add a Tuya device to my network but that is my personal preference.

I also limit custom components to ones that I have proven don't cause issues with the HA event loop. Easy way to rule them out is to run HA in safe mode for a while.

dmulcahey avatar Sep 27 '24 22:09 dmulcahey

I thought doing a recap could help. Now I understand I’m too old to keep playing with these toys. Goodbye.

Sorry if I misunderstood the "No way" comment you ended your reply with. I'm not trying to make you quit... just trying to explain that even though it feels like you have done everything doesn't mean there aren't more things to explore. In the thread you shared from Reddit there is even a user who pointed out the IKEA issue...

Device mix on the network (especially routing devices) is one of the most important factors for stability. There are a lot of devices on the market and some just don't perform well.

I'm not guaranteeing that this is the cause of your issue but it's an avenue worth exploring. With the time investment you have already committed I'd think you'd be willing to try this... and by your own account your devices are one of the few constants across your attempts to remediate your issue.

dmulcahey avatar Sep 27 '24 22:09 dmulcahey

IMO router devices are the most important consideration after the coordinator.

Pardon the ignorance, but what do you mean by router devices? I also have never heard of that coordinator. Better than my Sonoff?

ecchodun avatar Sep 27 '24 23:09 ecchodun

Router devices are generally mains powered devices that in addition to their user facing functionality also help route traffic for other devices on the network and they can also act as parents to end devices. Lots of bulbs, smart sockets, etc are routers. The device type can be seen on the device page in HA if you expand the zigbee details.

dmulcahey avatar Sep 28 '24 00:09 dmulcahey

Coordinator is a TubeZb SI MGM24 connected via USB

So you have one of the rarest out of stock devices out there and have no issues. Yeah, that's not going to help. BTW, another 3074 error tonight on a powered switch that never gave me errors before. Makes sense.

ecchodun avatar Sep 28 '24 06:09 ecchodun

Coordinator is a TubeZb SI MGM24 connected via USB

So you have one of the rarest out of stock devices out there and have no issues. Yeah, that's not going to help. BTW, another 3074 error tonight on a powered switch that never gave me errors before. Makes sense.

Honestly, I don't know how to help at this point. I get folks are frustrated but the commentary isn't helpful to anyone.

Just to give you another data point: I also have had this network running since ZHA was introduced and I ran it on a HUSBZB-1 for years... I know that won't make folks feel better but it is what it is.

In the visualization what is that switch connected to? For the router devices it shows connections to go to the device page, launch the manage device dialog, select the neighbors tab then click the title of the dialog so that the relationship column becomes available. See if you can determine what its parent is.

dmulcahey avatar Sep 28 '24 13:09 dmulcahey

In the visualization what is that switch connected to? For the router devices it shows connections to go to the device page, launch the manage device dialog, select the neighbors tab then click the title of the dialog so that the relationship column becomes available. See if you can determine what its parent is.

Thank you. Okay, so I now know the parent for the router device (a plug-in switch). It has an LQI of 57. What's next?

ecchodun avatar Sep 28 '24 14:09 ecchodun

And what's the model of the router (plug-in switch)? Also, please post your ZHA integration diagnostics.

TheJulianJES avatar Sep 28 '24 17:09 TheJulianJES

And what's the model of the router (plug-in switch)? Also, please post your ZHA integration diagnostics.

It's a S31 Lite zb by SONOFF. One of these - https://www.amazon.com/gp/product/B08Y87WD1X/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

I never (and I repeat never) had a 3074 error with this particular switch and it just popped up out of no where for an animation that was set to kick on at 1:00 a.m. early this morning.

My diagnostics are attached.

config_entry-zha-bee6cccfaff10b502497db0b3db8d06e.json

ecchodun avatar Sep 28 '24 17:09 ecchodun

Nothing?

ecchodun avatar Oct 06 '24 14:10 ecchodun

I also have the same issue :-(. The error appear without reason. I didn’t add any device or whatever.

Here I tried to switch off « escaliers » and error. And sometime no problem :-/ sometimes it’s others devices IMG_7569

Auka84 avatar Oct 08 '24 05:10 Auka84

Hi, I am experiencing the same kind of problems. Important note: Logic and reading of this and other threads suggests to me that it is actually NOT a zigpy / zha problem. So, actually off-topic. But since everything started to have problems since 2024.9.x (currently on 2024.10.0), I'm still posting it here. Just in case there still might be some change in recent zigpy/zha that could facilitate the issues.

I started to experience many 3074 problems throughout the network from 2024.9.x. Before, it was a stable network ; but - coincidentally with the September updates and the addition of 2 new Zigbee power plugs (bringing the total of my ZHA devices to 97), I started to get a lot of disconnected Devices.

I can't really tell if the problems are in ZHA or if this is just coincidental.

Right now, removing a few devices, things look a little bit more stable (the coordinator is a nabu casa, the one from HA), but particularly the TRVs (thermostats) would not respond to commands. Still, they do report their state correctly, also when turning their wheels. Following @dmulcahey 's tips, I went "after" my IKEA repeaters (TRADFRI Signal Repeater) and (instead of removing them or re-pairing them), went to the Device page in HA and clicked "reconfigure".

This did bring back (although delayed) control functionality on all my tuya TRVs, but did not so on my 2 Aqara. I do believe that the problem is more likely to be one in my radio/zigbee/device choice, but the strange thing is, that it all used to work very nicely before and suddenly became a debug nightmare (and partly, still is; some motion sensors for example will disconnect from time to time and need to be power cycled or re-paired to get "back in").

I'm attaching one of the log entries with stack trace, unsure whether it's just the expected output when devices do not "react".

I'm open to any suggestions / questions...

```

Logger: homeassistant.components.websocket_api.http.connection Source: components/websocket_api/commands.py:245 integration: Home Assistant WebSocket API (documentation, issues) First occurred: 1:18:49 AM (7 occurrences) Last logged: 11:52:05 AM

[546054599392] Unexpected exception [546126839472] Unexpected exception Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/zha/zigbee/cluster_handlers/init.py", line 67, in wrap_zigpy_exceptions yield File "/usr/local/lib/python3.12/site-packages/zha/zigbee/cluster_handlers/init.py", line 85, in wrapper return await RETRYABLE_REQUEST_DECORATOR(func)(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/zigpy/util.py", line 136, in retry return await func() ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/zhaquirks/tuya/init.py", line 776, in write_attributes await self.endpoint.tuya_manufacturer.write_attributes( File "/usr/local/lib/python3.12/site-packages/zhaquirks/tuya/init.py", line 521, in write_attributes await super().command( File "/usr/local/lib/python3.12/site-packages/zigpy/quirks/init.py", line 254, in command return await self.request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/zigpy/zcl/init.py", line 378, in request return await self._endpoint.request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/zigpy/endpoint.py", line 265, in request return await self.device.request( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/zigpy/device.py", line 334, in request await send_request() File "/usr/local/lib/python3.12/site-packages/zigpy/application.py", line 834, in request await self.send_packet( File "/usr/local/lib/python3.12/site-packages/bellows/zigbee/application.py", line 827, in send_packet raise zigpy.exceptions.DeliveryError( zigpy.exceptions.DeliveryError: Failed to deliver message: <sl_Status.ZIGBEE_DELIVERY_FAILED: 3074>

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/components/zha/helpers.py", line 1286, in handler return await func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/zha/climate.py", line 230, in async_set_temperature await self.entity_data.entity.async_set_temperature( File "/usr/local/lib/python3.12/site-packages/zha/application/platforms/climate/init.py", line 462, in async_set_temperature await self._thermostat_cluster_handler.async_set_heating_setpoint( File "/usr/local/lib/python3.12/site-packages/zha/zigbee/cluster_handlers/hvac.py", line 286, in async_set_heating_setpoint await self.write_attributes_safe({attr: temperature}) File "/usr/local/lib/python3.12/site-packages/zha/zigbee/cluster_handlers/init.py", line 614, in write_attributes_safe res = await self.write_attributes(attributes, manufacturer=manufacturer) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/zha/zigbee/cluster_handlers/init.py", line 84, in wrapper with wrap_zigpy_exceptions(): File "/usr/local/lib/python3.12/contextlib.py", line 158, in exit self.gen.throw(value) File "/usr/local/lib/python3.12/site-packages/zha/zigbee/cluster_handlers/init.py", line 76, in wrap_zigpy_exceptions raise ZHAException(message) from exc zha.exceptions.ZHAException: Failed to send request: Failed to deliver message: <sl_Status.ZIGBEE_DELIVERY_FAILED: 3074>

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 245, in handle_call_service response = await hass.services.async_call( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/core.py", line 2761, in async_call response_data = await coro ^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/core.py", line 2804, in _execute_service return await target(service_call) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 996, in entity_service_call single_response = await _handle_entity_call( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 1068, in _handle_entity_call result = await task ^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/climate/init.py", line 1029, in async_service_temperature_set await entity.async_set_temperature(**kwargs) File "/usr/src/homeassistant/homeassistant/components/zha/helpers.py", line 1288, in handler raise HomeAssistantError(err) from err homeassistant.exceptions.HomeAssistantError: Failed to send request: Failed to deliver message: <sl_Status.ZIGBEE_DELIVERY_FAILED: 3074>

</details>

lopezio avatar Oct 10 '24 14:10 lopezio

home-assistant_zha_2024-09-21T23-49-26.017Z.zip

Both files in Zip

Its hard to accept that this is all attributed to RF interference - Given all the different reports where everything is stable over long period of time.

i8nemo avatar Oct 10 '24 17:10 i8nemo