core ZHA devices unavailable and cannot re-pair after upgrade to 2023.3.5 from 2023.3.3

The problem

After upgrading to 2023.3.5 19 out of 56 aqara battery powered devices became unavailable. I tried to re-pair about 5 of them without any success. I've since reverted back to 2023.3.3 but the devices still remain unavailable and I cannot get them to add to the network. The only significant change was the update, prior to that I didn't have any problems with devices dropping off the network.

I have the Sonoff Zigbee 3.0 dongle. ZBDongle-E

What version of Home Assistant Core has the issue?

2023.3.5

What was the last working version of Home Assistant Core?

2023.3.3

What type of installation are you running?

Home Assistant OS

Integration causing the issue

ZHA

Link to integration documentation on our website

https://www.home-assistant.io/integrations/zha/

Diagnostics information

{ "home_assistant": { "installation_type": "Home Assistant OS", "version": "2023.3.3", "dev": false, "hassio": true, "virtualenv": false, "python_version": "3.10.10", "docker": true, "arch": "x86_64", "timezone": "Pacific/Honolulu", "os_name": "Linux", "os_version": "5.15.90", "supervisor": "2023.03.1", "host_os": "Home Assistant OS 9.5", "docker_version": "20.10.22", "chassis": "vm", "run_as_root": true }, "custom_components": { "hacs": { "version": "1.31.0", "requirements": [ "aiogithubapi>=22.10.1" ] }, "platerecognizer": { "version": "1.0.0", "requirements": [ "pillow", "requests" ] }, "alarmo": { "version": "v1.9.7", "requirements": [] }, "frigate": { "version": "3.0.1", "requirements": [] }, "webrtc": { "version": "v3.1.0", "requirements": [] }, "midea_ac_lan": { "version": "v0.3.16-Beta7", "requirements": [] } }, "integration_manifest": { "domain": "zha", "name": "Zigbee Home Automation", "after_dependencies": [ "onboarding", "usb" ], "codeowners": [ "@dmulcahey", "@adminiuga", "@puddly" ], "config_flow": true, "dependencies": [ "file_upload" ], "documentation": "https://www.home-assistant.io/integrations/zha", "iot_class": "local_polling", "loggers": [ "aiosqlite", "bellows", "crccheck", "pure_pcapy3", "zhaquirks", "zigpy", "zigpy_deconz", "zigpy_xbee", "zigpy_zigate", "zigpy_znp" ], "requirements": [ "bellows==0.34.9", "pyserial==3.5", "pyserial-asyncio==0.6", "zha-quirks==0.0.93", "zigpy-deconz==0.19.2", "zigpy==0.53.2", "zigpy-xbee==0.16.2", "zigpy-zigate==0.10.3", "zigpy-znp==0.9.3" ], "usb": [ { "vid": "10C4", "pid": "EA60", "description": "2652", "known_devices": [ "slae.sh cc2652rb stick" ] }, { "vid": "1A86", "pid": "55D4", "description": "sonoffplus*", "known_devices": [ "sonoff zigbee dongle plus v2" ] }, { "vid": "10C4", "pid": "EA60", "description": "sonoffplus*", "known_devices": [ "sonoff zigbee dongle plus" ] }, { "vid": "10C4", "pid": "EA60", "description": "tubeszb", "known_devices": [ "TubesZB Coordinator" ] }, { "vid": "1A86", "pid": "7523", "description": "tubeszb", "known_devices": [ "TubesZB Coordinator" ] }, { "vid": "1A86", "pid": "7523", "description": "zigstar", "known_devices": [ "ZigStar Coordinators" ] }, { "vid": "1CF1", "pid": "0030", "description": "conbee", "known_devices": [ "Conbee II" ] }, { "vid": "10C4", "pid": "8A2A", "description": "zigbee", "known_devices": [ "Nortek HUSBZB-1" ] }, { "vid": "0403", "pid": "6015", "description": "zigate", "known_devices": [ "ZiGate+" ] }, { "vid": "10C4", "pid": "EA60", "description": "zigate", "known_devices": [ "ZiGate" ] }, { "vid": "10C4", "pid": "8B34", "description": "bv 2010/10", "known_devices": [ "Bitron Video AV2010/10" ] } ], "zeroconf": [ { "type": "_esphomelib._tcp.local.", "name": "tube*" }, { "type": "_zigate-zigbee-gateway._tcp.local.", "name": "zigate" }, { "type": "_zigstar_gw._tcp.local.", "name": "zigstar" }, { "type": "_slzb-06._tcp.local.", "name": "slzb-06*" } ], "is_built_in": true }, "data": { "ieee": "REDACTED", "nwk": 0, "manufacturer": "Silicon Labs", "model": "EZSP", "name": "Silicon Labs EZSP", "quirk_applied": false, "quirk_class": "bellows.zigbee.device.EZSPCoordinator", "manufacturer_code": 43981, "power_source": "Mains", "lqi": 255, "rssi": 0, "last_seen": "2023-03-19T23:02:32", "available": true, "device_type": "Coordinator", "signature": { "node_descriptor": "NodeDescriptor(logical_type=<LogicalType.Coordinator: 0>, complex_descriptor_available=0, user_descriptor_available=0, reserved=0, aps_flags=0, frequency_band=<FrequencyBand.Freq2400MHz: 8>, mac_capability_flags=<MACCapabilityFlags.AllocateAddress|RxOnWhenIdle|MainsPowered|FullFunctionDevice|AlternatePanCoordinator: 143>, manufacturer_code=43981, maximum_buffer_size=82, maximum_incoming_transfer_size=128, server_mask=11329, maximum_outgoing_transfer_size=128, descriptor_capability_field=<DescriptorCapability.NONE: 0>, *allocate_address=True, *is_alternate_pan_coordinator=True, *is_coordinator=True, *is_end_device=False, *is_full_function_device=True, *is_mains_powered=True, *is_receiver_on_when_idle=True, *is_router=False, *is_security_capable=False)", "endpoints": { "1": { "profile_id": 260, "device_type": "0x0400", "in_clusters": [ "0x0000", "0x0006", "0x000a", "0x0019", "0x0501" ], "out_clusters": [ "0x0001", "0x0020", "0x0500", "0x0502" ] } } }, "active_coordinator": true, "entities": [], "neighbors": [ { "device_type": "Router", "rx_on_when_idle": "On", "relationship": "Sibling", "extended_pan_id": "REDACTED", "ieee": "REDACTED", "nwk": "0x0FDB", "permit_joining": "Unknown", "depth": "15", "lqi": "244" }, { "device_type": "Router", "rx_on_when_idle": "On", "relationship": "Sibling", "extended_pan_id": "REDACTED", "ieee": "REDACTED", "nwk": "0x4154", "permit_joining": "Unknown", "depth": "15", "lqi": "140" }, { "device_type": "Router", "rx_on_when_idle": "On", "relationship": "Sibling", "extended_pan_id": "REDACTED", "ieee": "REDACTED", "nwk": "0x4C31", "permit_joining": "Unknown", "depth": "15", "lqi": "70" }, { "device_type": "Router", "rx_on_when_idle": "On", "relationship": "Sibling", "extended_pan_id": "REDACTED", "ieee": "REDACTED", "nwk": "0x7E7B", "permit_joining": "Unknown", "depth": "15", "lqi": "162" }, { "device_type": "Router", "rx_on_when_idle": "On", "relationship": "Sibling", "extended_pan_id": "REDACTED", "ieee": "REDACTED", "nwk": "0x98BF", "permit_joining": "Unknown", "depth": "15", "lqi": "140" }, { "device_type": "Router", "rx_on_when_idle": "On", "relationship": "Sibling", "extended_pan_id": "REDACTED", "ieee": "REDACTED", "nwk": "0xA28E", "permit_joining": "Unknown", "depth": "15", "lqi": "191" }, { "device_type": "Router", "rx_on_when_idle": "On", "relationship": "Sibling", "extended_pan_id": "REDACTED", "ieee": "REDACTED", "nwk": "0xB664", "permit_joining": "Unknown", "depth": "15", "lqi": "168" }, { "device_type": "Router", "rx_on_when_idle": "On", "relationship": "Sibling", "extended_pan_id": "REDACTED", "ieee": "REDACTED", "nwk": "0xC8F1", "permit_joining": "Unknown", "depth": "15", "lqi": "145" }, { "device_type": "Router", "rx_on_when_idle": "On", "relationship": "Sibling", "extended_pan_id": "REDACTED", "ieee": "REDACTED", "nwk": "0xCE9B", "permit_joining": "Unknown", "depth": "15", "lqi": "252" }, { "device_type": "EndDevice", "rx_on_when_idle": "Off", "relationship": "Child", "extended_pan_id": "REDACTED", "ieee": "REDACTED", "nwk": "0xB58C", "permit_joining": "NotAccepting", "depth": "1", "lqi": "185" }, { "device_type": "EndDevice", "rx_on_when_idle": "Off", "relationship": "Child", "extended_pan_id": "REDACTED", "ieee": "REDACTED", "nwk": "0xBC93", "permit_joining": "NotAccepting", "depth": "1", "lqi": "248" }, { "device_type": "EndDevice", "rx_on_when_idle": "Off", "relationship": "Child", "extended_pan_id": "REDACTED", "ieee": "REDACTED", "nwk": "0x29EF", "permit_joining": "NotAccepting", "depth": "1", "lqi": "220" } ], "routes": [ { "dest_nwk": "0xB664", "route_status": "Active", "memory_constrained": true, "many_to_one": true, "route_record_required": true, "next_hop": "0xB664" }, { "dest_nwk": "0xCE9B", "route_status": "Active", "memory_constrained": false, "many_to_one": false, "route_record_required": false, "next_hop": "0xCE9B" }, { "dest_nwk": "0x4C31", "route_status": "Active", "memory_constrained": false, "many_to_one": false, "route_record_required": false, "next_hop": "0x4C31" }, { "dest_nwk": "0x7E7B", "route_status": "Active", "memory_constrained": false, "many_to_one": false, "route_record_required": false, "next_hop": "0x7E7B" }, { "dest_nwk": "0x0FDB", "route_status": "Active", "memory_constrained": true, "many_to_one": true, "route_record_required": true, "next_hop": "0x0FDB" }, { "dest_nwk": "0x1348", "route_status": "Active", "memory_constrained": false, "many_to_one": false, "route_record_required": false, "next_hop": "0x98BF" }, { "dest_nwk": "0x4154", "route_status": "Active", "memory_constrained": true, "many_to_one": true, "route_record_required": true, "next_hop": "0xB664" }, { "dest_nwk": "0x98BF", "route_status": "Active", "memory_constrained": true, "many_to_one": true, "route_record_required": true, "next_hop": "0xCE9B" }, { "dest_nwk": "0xA377", "route_status": "Active", "memory_constrained": false, "many_to_one": false, "route_record_required": false, "next_hop": "0x98BF" }, { "dest_nwk": "0xB627", "route_status": "Active", "memory_constrained": false, "many_to_one": false, "route_record_required": false, "next_hop": "0xC8F1" }, { "dest_nwk": "0xC8F1", "route_status": "Active", "memory_constrained": false, "many_to_one": false, "route_record_required": false, "next_hop": "0xC8F1" } ], "endpoint_names": [ { "name": "IAS_CONTROL" } ], "user_given_name": null, "device_reg_id": "6daff66f1168e5d43bf806885774435f", "area_id": "office", "cluster_details": { "1": { "device_type": { "name": "IAS_CONTROL", "id": 1024 }, "profile_id": 260, "in_clusters": { "0x0000": { "endpoint_attribute": "basic", "attributes": {}, "unsupported_attributes": {} }, "0x0006": { "endpoint_attribute": "on_off", "attributes": {}, "unsupported_attributes": {} }, "0x000a": { "endpoint_attribute": "time", "attributes": {}, "unsupported_attributes": {} }, "0x0019": { "endpoint_attribute": "ota", "attributes": {}, "unsupported_attributes": {} }, "0x0501": { "endpoint_attribute": "ias_ace", "attributes": {}, "unsupported_attributes": {} } }, "out_clusters": { "0x0001": { "endpoint_attribute": "power", "attributes": {}, "unsupported_attributes": {} }, "0x0020": { "endpoint_attribute": "poll_control", "attributes": {}, "unsupported_attributes": {} }, "0x0500": { "endpoint_attribute": "ias_zone", "attributes": {}, "unsupported_attributes": {} }, "0x0502": { "endpoint_attribute": "ias_wd", "attributes": {}, "unsupported_attributes": {} } } } } } }

Example YAML snippet

zha:
  zigpy_config:
    network:
      channel: 25             # What channel the radio should try to use.
      channels: [15, 20, 25]

Anything in the logs that might be useful for us?

Here's zha debug logs
https://gist.github.com/sea3pea0/190ff79add39ac459c10ebea1a0d8af6

Additional information

No response

Mar 20 '23 10:03 sea3pea0

Hey there @dmulcahey, @adminiuga, @puddly, mind taking a look at this issue as it has been labeled with an integration (zha) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of zha can trigger bot actions by commenting:

@home-assistant close Closes the issue.
@home-assistant rename Awesome new title Renames the issue.
@home-assistant reopen Reopen the issue.
@home-assistant unassign zha Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

zha documentation zha source _{^{(message by IssueLinks)}}

Mar 20 '23 10:03 home-assistant[bot]

Here's my ZHA debug logs

Mar 20 '23 10:03 sea3pea0

ZHA unfortunately does not have control over device connectivity. The only effect an update would have is restarting the stack.

If you cannot join new devices, this may point to a broader problem. Take a look at the general "connectivity issues" guide and see if you can follow any steps to potentially help out.

Mar 20 '23 14:03 puddly

I'm able to add new devices. I had a brand new device laying around to test with. I just can't add the ones that dropped off after I experienced the problem. Those devices behave differently now too. They don't have the same blink pattern after holding the pairing button

Mar 20 '23 16:03 sea3pea0

I managed to get the sensors to join again. Not sure what's made them more stubborn, but it became more difficult to get them to pair lately. Maybe some new 2.4ghtz interference from a neighbor or something..

Mar 20 '23 23:03 sea3pea0