integration fails to initialize with Phoscon Raspbee II

[joshkephart]

The problem

The ZHA integration fails initialize after the 2023.12.0 core update (and persisting in 12.1).

What version of Home Assistant Core has the issue?

core-2023.12.1

What was the last working version of Home Assistant Core?

core-2023.11.3

What type of installation are you running?

Home Assistant OS

Integration causing the issue

ZHA

Link to integration documentation on our website

https://www.home-assistant.io/integrations/zha/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

2023-12-10 09:56:53.061 WARNING (MainThread) [zigpy_deconz.api] No response to 'CommandId.read_parameter' command with seq id '0x01'
2023-12-10 09:56:59.255 WARNING (MainThread) [zigpy_deconz.api] No response to 'CommandId.read_parameter' command with seq id '0x01'
2023-12-10 09:57:04.749 WARNING (MainThread) [zigpy_deconz.api] No response to 'CommandId.read_parameter' command with seq id '0x01'
2023-12-10 09:57:04.862 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry ttyAMA0, s/n: n/a for zha
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 559, in _command
    return await fut
           ^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 402, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/__init__.py", line 167, in async_setup_entry
    zha_gateway = await ZHAGateway.async_from_config(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/core/gateway.py", line 192, in async_from_config
    await instance.async_initialize()
  File "/usr/src/homeassistant/homeassistant/components/zha/core/gateway.py", line 210, in async_initialize
    await app.startup(auto_form=True)
  File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 223, in startup
    await self.connect()
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/zigbee/application.py", line 95, in connect
    await api.connect()
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 456, in connect
    await self.version()
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 733, in version
    self._protocol_version = await self.read_parameter(
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 752, in read_parameter
    rsp = await self._command(
          ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 558, in _command
    async with asyncio_timeout(COMMAND_TIMEOUT):
  File "/usr/local/lib/python3.11/asyncio/timeouts.py", line 111, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2023-12-10 10:01:35.476 DEBUG (MainThread) [homeassistant.components.zha] Couldn't start coordinator (attempt 2 of 3)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 559, in _command
    return await fut
           ^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/zha/__init__.py", line 167, in async_setup_entry
    zha_gateway = await ZHAGateway.async_from_config(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/core/gateway.py", line 192, in async_from_config
    await instance.async_initialize()
  File "/usr/src/homeassistant/homeassistant/components/zha/core/gateway.py", line 210, in async_initialize
    await app.startup(auto_form=True)
  File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 223, in startup
    await self.connect()
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/zigbee/application.py", line 95, in connect
    await api.connect()
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 456, in connect
    await self.version()
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 733, in version
    self._protocol_version = await self.read_parameter(
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 752, in read_parameter
    rsp = await self._command(
          ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 558, in _command
    async with asyncio_timeout(COMMAND_TIMEOUT):
  File "/usr/local/lib/python3.11/asyncio/timeouts.py", line 111, in __aexit__
    raise TimeoutError from exc_val
2023-12-10 10:01:39.798 DEBUG (MainThread) [zigpy_deconz.uart] Connecting to /dev/ttyAMA0
2023-12-10 10:01:39.798 DEBUG (MainThread) [zigpy.serial] Opening a serial connection to '/dev/ttyAMA0' (115200 baudrate)
2023-12-10 10:01:39.802 DEBUG (MainThread) [zigpy_deconz.uart] Connection made
2023-12-10 10:01:39.803 DEBUG (MainThread) [zigpy_deconz.uart] Connected to /dev/ttyAMA0
2023-12-10 10:01:39.803 DEBUG (MainThread) [zigpy_deconz.api] Sending CommandId.read_parameter{'parameter_id': <NetworkParameter.protocol_version: 34>, 'parameter': b''} (seq=1)
2023-12-10 10:01:39.804 DEBUG (MainThread) [zigpy_deconz.uart] Send: 0a01000800010022
2023-12-10 10:01:41.605 WARNING (MainThread) [zigpy_deconz.api] No response to 'CommandId.read_parameter' command with seq id '0x01'
2023-12-10 10:01:41.609 DEBUG (Thread-22) [aiosqlite] executing functools.partial(<function PersistingListener._set_isolation_level.<locals>.<lambda> at 0x7f59f23b00>)
2023-12-10 10:01:41.609 DEBUG (Thread-22) [aiosqlite] operation functools.partial(<function PersistingListener._set_isolation_level.<locals>.<lambda> at 0x7f59f23b00>) completed
2023-12-10 10:01:41.612 DEBUG (MainThread) [zigpy_deconz.api] Serial '/dev/ttyAMA0' connection lost unexpectedly: None
2023-12-10 10:01:41.613 DEBUG (Thread-22) [aiosqlite] executing functools.partial(<built-in method execute of sqlite3.Connection object at 0x7f5a594220>, 'PRAGMA wal_checkpoint;', [])
2023-12-10 10:01:41.614 DEBUG (Thread-22) [aiosqlite] operation functools.partial(<built-in method execute of sqlite3.Connection object at 0x7f5a594220>, 'PRAGMA wal_checkpoint;', []) completed
2023-12-10 10:01:41.616 DEBUG (Thread-22) [aiosqlite] executing functools.partial(<function PersistingListener._set_isolation_level.<locals>.<lambda> at 0x7f59f225c0>)
2023-12-10 10:01:41.616 DEBUG (Thread-22) [aiosqlite] operation functools.partial(<function PersistingListener._set_isolation_level.<locals>.<lambda> at 0x7f59f225c0>) completed
2023-12-10 10:01:41.617 DEBUG (Thread-22) [aiosqlite] executing functools.partial(<built-in method close of sqlite3.Connection object at 0x7f5a594220>)
2023-12-10 10:01:41.621 DEBUG (Thread-22) [aiosqlite] operation functools.partial(<built-in method close of sqlite3.Connection object at 0x7f5a594220>) completed
2023-12-10 10:01:41.723 DEBUG (MainThread) [homeassistant.components.zha] Couldn't start coordinator (attempt 3 of 3)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 559, in _command
    return await fut
           ^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/zha/__init__.py", line 167, in async_setup_entry
    zha_gateway = await ZHAGateway.async_from_config(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/core/gateway.py", line 192, in async_from_config
    await instance.async_initialize()
  File "/usr/src/homeassistant/homeassistant/components/zha/core/gateway.py", line 210, in async_initialize
    await app.startup(auto_form=True)
  File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 223, in startup
    await self.connect()
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/zigbee/application.py", line 95, in connect
    await api.connect()
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 456, in connect
    await self.version()
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 733, in version
    self._protocol_version = await self.read_parameter(
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 752, in read_parameter
    rsp = await self._command(
          ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 558, in _command
    async with asyncio_timeout(COMMAND_TIMEOUT):
  File "/usr/local/lib/python3.11/asyncio/timeouts.py", line 111, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2023-12-10 10:01:41.730 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry ttyAMA0, s/n: n/a for zha
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 559, in _command
    return await fut
           ^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 402, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/__init__.py", line 167, in async_setup_entry
    zha_gateway = await ZHAGateway.async_from_config(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/core/gateway.py", line 192, in async_from_config
    await instance.async_initialize()
  File "/usr/src/homeassistant/homeassistant/components/zha/core/gateway.py", line 210, in async_initialize
    await app.startup(auto_form=True)
  File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 223, in startup
    await self.connect()
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/zigbee/application.py", line 95, in connect
    await api.connect()
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 456, in connect
    await self.version()
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 733, in version
    self._protocol_version = await self.read_parameter(
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 752, in read_parameter
    rsp = await self._command(
          ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 558, in _command
    async with asyncio_timeout(COMMAND_TIMEOUT):
  File "/usr/local/lib/python3.11/asyncio/timeouts.py", line 111, in __aexit__
    raise TimeoutError from exc_val
TimeoutError

Additional information

If I reload the integration from the integrations page, it will fail to initialize, however, if I reconfigure the integration and go through all the set up steps again (while keeping my radio network), it will set up successfully and remain operational for a seemingly random amount of time (sometimes a few minutes, sometimes an hour or more). Eventually I'll return to all my zigbee devices being unavailable and the integration back in the failed to initialize state.

I'm using a Phoscon Raspbee II card.

Downgrading to 2023.11.3 resolves the issue.

[home-assistant[bot]]

Hey there @dmulcahey, @adminiuga, @puddly, @thejulianjes, mind taking a look at this issue as it has been labeled with an integration (zha) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of zha can trigger bot actions by commenting:

• @home-assistant close Closes the issue.
• @home-assistant rename Awesome new title Renames the issue.
• @home-assistant reopen Reopen the issue.
• @home-assistant unassign zha Removes the current integration label and assignees on the issue, add the integration domain after the command.
• @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
• @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

---

zha documentation zha source _{^{(message by IssueLinks)}}

[puddly]

Can you include a complete debug log (https://www.home-assistant.io/integrations/zha/#debug-logging) of the integration failing to set up, you re-configuring it, and then it failing after an hour? Re-configuring the serial port shouldn't be making any difference, as the new serial port path will match the old.

[cvermot]

Hi, Same problem here, with Conbee 2 USB stick.

We can reload the ZHA intergration via Configure option. Zigbee devices works for somes minutes or hours... but the integration seems to lost his configuration after some times.

The problems has appeard some days ago (possibily with the last 2023.11 release ?). No more success after updating HA OS.

Here are my versions : Core 2023.12.1 Supervisor 2023.11.6 Operating System 11.2 Interface utilisateur 20231208.2

Problem seems duplicated here : https://github.com/home-assistant/core/issues/105445

[joshkephart]

Can you include a complete debug log (https://www.home-assistant.io/integrations/zha/#debug-logging) of the integration failing to set up, you re-configuring it, and then it failing after an hour? Re-configuring the serial port shouldn't be making any difference, as the new serial port path will match the old.

Yeah I'll get one uploaded in the next day or two. I had rolled back to 11.3 to get my system working again after creating this issue. I'll temporarily upgrade again, get a debug log of the integration starting up / failing to initialize and post it.

[bsfaxi]

Hi, I have the same issue here. It happens on my installation and also on another installation that I did for my friend! Same RPI4, 64b, with 4 GB of RAM and a Raspbee II (shield). The same error happens once a day, or every 2 days... A reload of the ZHA integration does repair temporarily, but not permanently. :( It started since release 2023.12. How can I help in investigation?

[joshkephart]

FYI @puddly - I have not uploaded a debug log because I have not been able to reproduce the issue again for some reason. There was only one change between these recent attempts and consistent behavior before I wrote this issue. Here's how I got to this state:

• upgraded to 2023.12.0 and got the original failures and behaviors reported in this issue
• upgraded to 2023.12.1, no change, opened this issue
• restored ha-core from 2023.11.3 partial backup to downgrade to a working state
• rebooted and zha was happy
• wrote this issue
• replaced batteries in about 5 zigbee sensors I have that were all reporting unavailable
• upgraded to 12.1 again to capture a debug log for this issue
• zha is now starting up fine, I cannot get it to fail anymore

I noticed you mentioned all these recent zha issues in a couple recent PRs related to a new zigpy watchdog timer. I'm wondering if my issue may have popped up due to a longer than usual zigbee initialization caused by a decent number of offline devices in my zigbee network. Now that everything is online it starts up faster and is not tripping the watchdog? Just a theory based on my info and my inability to reproduce it now. Unfortunately I can't provide any more debug info since it's not happening for me anymore. I'm stable and operational on 2023.12.1 for now.

Hope that little bit of extra info helps!

[bsfaxi]

Thanks Joshua for your inputs. Lucky you! :) On my side, I still have the issue on my installation. I upgraded this morning to 2023.12.2, and will monitor that. On the installation of my friend, I've downgraded to 2023.11.3, and it's working like a charm. He has no batteries based devices on his zigbee network! I have a debug file generated before yesterday. I'll check it and upload it.

[bsfaxi]

Please find, attached in the bottom below, 2 logs: one without debug, the other with debug.

All seems to start after some communication failures:

2023-12-13 04:37:25.286 WARNING (MainThread) [homeassistant.components.zha.core.cluster_handlers] [0x6509:11:0x0006]: async_initialize: all attempts have failed: [DeliveryError('Failed to deliver packet: <TXStatus.NWK_ROUTE_DISCOVERY_FAILED: 208>'), TimeoutError(), TimeoutError(), TimeoutError()]

Then, 3 tentatives, then the ZHA is broken:

2023-12-13 04:37:27.089 WARNING (MainThread) [zigpy.application] Watchdog failure
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 559, in _command
    return await fut
           ^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 656, in _watchdog_loop
    await self._watchdog_feed()
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/zigbee/application.py", line 81, in _watchdog_feed
    await self._api.get_device_state()
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 829, in get_device_state
    rsp = await self._command(CommandId.device_state)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 558, in _command
    async with asyncio_timeout(COMMAND_TIMEOUT):
  File "/usr/local/lib/python3.11/asyncio/timeouts.py", line 111, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2023-12-13 04:37:27.097 DEBUG (MainThread) [zigpy.application] Connection to the radio has been lost: TimeoutError()
2023-12-13 04:37:27.098 DEBUG (MainThread) [zigpy.application] Stopping watchdog loop

...

2023-12-13 04:37:34.949 DEBUG (MainThread) [zigpy.serial] Opening a serial connection to '/dev/ttyAMA0' (38400 baudrate)
2023-12-13 04:37:36.754 WARNING (MainThread) [zigpy_deconz.api] No response to 'CommandId.read_parameter' command with seq id '0x01'
2023-12-13 04:37:36.756 DEBUG (Thread-875) [aiosqlite] executing functools.partial(<function PersistingListener._set_isolation_level.<locals>.<lambda> at 0x7f4a512660>)
2023-12-13 04:37:36.756 DEBUG (Thread-875) [aiosqlite] operation functools.partial(<function PersistingListener._set_isolation_level.<locals>.<lambda> at 0x7f4a512660>) completed
2023-12-13 04:37:36.759 DEBUG (Thread-875) [aiosqlite] executing functools.partial(<built-in method execute of sqlite3.Connection object at 0x7f5dc7ce50>, 'PRAGMA wal_checkpoint;', [])
2023-12-13 04:37:36.759 DEBUG (Thread-875) [aiosqlite] operation functools.partial(<built-in method execute of sqlite3.Connection object at 0x7f5dc7ce50>, 'PRAGMA wal_checkpoint;', []) completed
2023-12-13 04:37:36.761 DEBUG (Thread-875) [aiosqlite] executing functools.partial(<function PersistingListener._set_isolation_level.<locals>.<lambda> at 0x7f4a5125c0>)
2023-12-13 04:37:36.762 DEBUG (Thread-875) [aiosqlite] operation functools.partial(<function PersistingListener._set_isolation_level.<locals>.<lambda> at 0x7f4a5125c0>) completed
2023-12-13 04:37:36.763 DEBUG (Thread-875) [aiosqlite] executing functools.partial(<built-in method close of sqlite3.Connection object at 0x7f5dc7ce50>)
2023-12-13 04:37:36.767 DEBUG (Thread-875) [aiosqlite] operation functools.partial(<built-in method close of sqlite3.Connection object at 0x7f5dc7ce50>) completed
2023-12-13 04:37:36.868 DEBUG (MainThread) [homeassistant.components.zha] Couldn't start coordinator (attempt 1 of 3)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 559, in _command
    return await fut
           ^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/zha/__init__.py", line 167, in async_setup_entry
    zha_gateway = await ZHAGateway.async_from_config(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/core/gateway.py", line 192, in async_from_config
    await instance.async_initialize()
  File "/usr/src/homeassistant/homeassistant/components/zha/core/gateway.py", line 210, in async_initialize
    await app.startup(auto_form=True)
  File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 223, in startup
    await self.connect()
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/zigbee/application.py", line 95, in connect
    await api.connect()
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 456, in connect
    await self.version()
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 733, in version
    self._protocol_version = await self.read_parameter(
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 752, in read_parameter
    rsp = await self._command(
          ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/zigpy_deconz/api.py", line 558, in _command
    async with asyncio_timeout(COMMAND_TIMEOUT):
  File "/usr/local/lib/python3.11/asyncio/timeouts.py", line 111, in __aexit__
    raise TimeoutError from exc_val
TimeoutError

zha_error_debug_20231213_4am.txt zha_error_20231211_6pm.txt

[bsfaxi]

Just adding feedback after migrating to 2023.12.2: it works without issues since 2 days now! I'll also migrate to 2023.12.3 and give feedback too.

[bsfaxi]

Just for follow-up, ZHA is still working since 2 days after upgrade to 2023.12.3.

[cvermot]

Hi,

I updated to 2023.12.3. It work after reboot. I will keep you informed about any new disruption on my zigbee networks.

[bsfaxi]

Hi, Just to let you know that it works since last week without any problem.

[issue-triage-workflows[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.