core Unhandled Exceptions in light turn on and off service

The problem

After updating to 2024.10.1 I started getting a lot of Failed to deliver message: <sl_Status.ZIGBEE_DELIVERY_FAILED: 3074> errors and my lights would not respond to commands. I noticed the Ikea E14 opal 400lm lightbulb closest to my skyconnect coordinator had dropped out, so I tried repairing it but it did not help. I downgraded to 2024.10.0 again, but after downgrading that E14 lightbulb will not repair, it gets to the "configuring" state when resetting it and entering pairing mode, but it will never complete the configuring state when searching for new zigbee device in HA. I still have the same 3074 errors on 2024.10.0 now and my Zigbee network is not working anymore after having been stable for many months. I tried removing the E14 bulb entity completely in hopes the zigbee mesh would disregard it and repair itself but it has not helped.

I was able to find the following in the logs:

`Logger: homeassistant.components.websocket_api.http.connection Source: components/websocket_api/commands.py:245 integration: Home Assistant WebSocket API (documentation, issues) First occurred: 07:11:17 (14 occurrences) Last logged: 09:41:52

[140462754609856] Unexpected exception [140461825007552] Unexpected exception [140461449062016] Unexpected exception [140461515628336] Unexpected exception [140461931824608] Unexpected exception Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 245, in handle_call_service response = await hass.services.async_call( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/core.py", line 2761, in async_call response_data = await coro ^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/core.py", line 2804, in _execute_service return await target(service_call) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/adaptive_lighting/hass_utils.py", line 62, in service_func_proxy await existing_service.job.target(call) File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 996, in entity_service_call single_response = await _handle_entity_call( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 1068, in _handle_entity_call result = await task ^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/light/init.py", line 626, in async_handle_light_on_service await async_handle_light_off_service(light, call) File "/usr/src/homeassistant/homeassistant/components/light/init.py", line 639, in async_handle_light_off_service await light.async_turn_off(**filter_turn_off_params(light, params)) File "/usr/src/homeassistant/homeassistant/components/group/light.py", line 194, in async_turn_off await self.hass.services.async_call( File "/usr/src/homeassistant/homeassistant/core.py", line 2761, in async_call response_data = await coro ^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/core.py", line 2804, in _execute_service return await target(service_call) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 1021, in entity_service_call raise result from None File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1610, in async_request_call return await coro ^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 1068, in _handle_entity_call result = await task ^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/light/init.py", line 639, in async_handle_light_off_service await light.async_turn_off(**filter_turn_off_params(light, params)) File "/usr/src/homeassistant/homeassistant/components/zha/helpers.py", line 1288, in handler raise HomeAssistantError(err) from err homeassistant.exceptions.HomeAssistantError: Failed to send request: Failed to deliver message: <sl_Status.ZIGBEE_DELIVERY_FAILED: 3074>`

What version of Home Assistant Core has the issue?

2024.10.1

What was the last working version of Home Assistant Core?

2024.10.0

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Home Assistant WebSocket API

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Logger: homeassistant.components.websocket_api.http.connection
Source: components/websocket_api/commands.py:245
integration: Home Assistant WebSocket API ([documentation](https://www.home-assistant.io/integrations/websocket_api), [issues](https://github.com/home-assistant/core/issues?q=is%3Aissue+is%3Aopen+label%3A%22integration%3A+websocket_api%22))
First occurred: 07:11:17 (14 occurrences)
Last logged: 09:41:52

[140462754609856] Unexpected exception
[140461825007552] Unexpected exception
[140461449062016] Unexpected exception
[140461515628336] Unexpected exception
[140461931824608] Unexpected exception
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 245, in handle_call_service
    response = await hass.services.async_call(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2761, in async_call
    response_data = await coro
                    ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2804, in _execute_service
    return await target(service_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/adaptive_lighting/hass_utils.py", line 62, in service_func_proxy
    await existing_service.job.target(call)
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 996, in entity_service_call
    single_response = await _handle_entity_call(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 1068, in _handle_entity_call
    result = await task
             ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/light/__init__.py", line 626, in async_handle_light_on_service
    await async_handle_light_off_service(light, call)
  File "/usr/src/homeassistant/homeassistant/components/light/__init__.py", line 639, in async_handle_light_off_service
    await light.async_turn_off(**filter_turn_off_params(light, params))
  File "/usr/src/homeassistant/homeassistant/components/group/light.py", line 194, in async_turn_off
    await self.hass.services.async_call(
  File "/usr/src/homeassistant/homeassistant/core.py", line 2761, in async_call
    response_data = await coro
                    ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2804, in _execute_service
    return await target(service_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 1021, in entity_service_call
    raise result from None
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1610, in async_request_call
    return await coro
           ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 1068, in _handle_entity_call
    result = await task
             ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/light/__init__.py", line 639, in async_handle_light_off_service
    await light.async_turn_off(**filter_turn_off_params(light, params))
  File "/usr/src/homeassistant/homeassistant/components/zha/helpers.py", line 1288, in handler
    raise HomeAssistantError(err) from err
homeassistant.exceptions.HomeAssistantError: Failed to send request: Failed to deliver message: <sl_Status.ZIGBEE_DELIVERY_FAILED: 3074>

Additional information

No response

Oct 07 '24 08:10 mathiasolofsson

Hey there @home-assistant/core, mind taking a look at this issue as it has been labeled with an integration (websocket_api) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of websocket_api can trigger bot actions by commenting:

@home-assistant close Closes the issue.
@home-assistant rename Awesome new title Renames the issue.
@home-assistant reopen Reopen the issue.
@home-assistant unassign websocket_api Removes the current integration label and assignees on the issue, add the integration domain after the command.
@home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
@home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

websocket_api documentation websocket_api source _{^{(message by IssueLinks)}}

Oct 07 '24 08:10 home-assistant[bot]

Also found this in the logs, not sure if it's related:

`Logger: homeassistant Source: /usr/src/homeassistant/homeassistant/runner.py:147 First occurred: 08:52:16 (1 occurrences) Last logged: 08:52:16

Error doing job: Task exception was never retrieved (None) Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/zha/zigbee/cluster_handlers/init.py", line 67, in wrap_zigpy_exceptions yield File "/usr/local/lib/python3.12/site-packages/zha/zigbee/cluster_handlers/init.py", line 85, in wrapper return await RETRYABLE_REQUEST_DECORATOR(func)(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/zigpy/util.py", line 136, in retry return await func() ^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/zigpy/zcl/init.py", line 378, in request return await self._endpoint.request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/zigpy/endpoint.py", line 265, in request return await self.device.request( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/zigpy/device.py", line 339, in request await send_request() File "/usr/local/lib/python3.12/site-packages/zigpy/application.py", line 834, in request await self.send_packet( File "/usr/local/lib/python3.12/site-packages/bellows/zigbee/application.py", line 827, in send_packet raise zigpy.exceptions.DeliveryError( zigpy.exceptions.DeliveryError: Failed to deliver message: <sl_Status.ZIGBEE_DELIVERY_FAILED: 3074>

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/local/lib/python3.12/site-packages/zha/application/gateway.py", line 607, in async_device_initialized await self._async_device_rejoined(zha_device) File "/usr/local/lib/python3.12/site-packages/zha/application/gateway.py", line 648, in _async_device_rejoined await zha_device.async_configure() File "/usr/local/lib/python3.12/site-packages/zha/zigbee/device.py", line 745, in async_configure await self.identify_ch.trigger_effect( File "/usr/local/lib/python3.12/site-packages/zha/zigbee/cluster_handlers/init.py", line 84, in wrapper with wrap_zigpy_exceptions(): File "/usr/local/lib/python3.12/contextlib.py", line 158, in exit self.gen.throw(value) File "/usr/local/lib/python3.12/site-packages/zha/zigbee/cluster_handlers/init.py", line 76, in wrap_zigpy_exceptions raise ZHAException(message) from exc zha.exceptions.ZHAException: Failed to send request: Failed to deliver message: <sl_Status.ZIGBEE_DELIVERY_FAILED: 3074>`

Oct 07 '24 08:10 mathiasolofsson

Shut off the main circuit breaker to the whole house for a minute, and then on again to make all devices unpowered for a while. After about 1hr after switching on the circuit breaker the zigbee network now works again and seems stable so mustve been a conflict somewhere in the zigbee network or a device that had locked up.

Oct 07 '24 20:10 mathiasolofsson

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

Jan 05 '25 22:01 issue-triage-workflows[bot]

Hey there @dmulcahey, @adminiuga, @puddly, @thejulianjes, mind taking a look at this issue as it has been labeled with an integration (zha) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of zha can trigger bot actions by commenting:

@home-assistant close Closes the issue.
@home-assistant rename Awesome new title Renames the issue.
@home-assistant reopen Reopen the issue.
@home-assistant unassign zha Removes the current integration label and assignees on the issue, add the integration domain after the command.
@home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
@home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

zha documentation zha source _{^{(message by IssueLinks)}}

Jan 06 '25 03:01 home-assistant[bot]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

Apr 06 '25 04:04 issue-triage-workflows[bot]

@mathiasolofsson Has your network been stable since you powered everything down?

Apr 07 '25 06:04 chmac

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

Jul 06 '25 08:07 issue-triage-workflows[bot]