zigpy-znp
zigpy-znp copied to clipboard
Error doing job: Task exception was never retrieved
I've been getting the following exceptions from zigpy_znp, using a zzh! in Home Assistant 2021.6.4:
Logger: homeassistant
Source: /usr/src/homeassistant/homeassistant/runner.py:97
First occurred: 4:55:17 PM (5 occurrences)
Last logged: 4:56:33 PM
Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/zigpy_znp/api.py", line 563, in request_callback_rsp
return await callback_rsp
asyncio.exceptions.CancelledError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/zigpy_znp/api.py", line 563, in request_callback_rsp
return await callback_rsp
File "/usr/local/lib/python3.8/site-packages/async_timeout/__init__.py", line 55, in __aexit__
self._do_exit(exc_type)
File "/usr/local/lib/python3.8/site-packages/async_timeout/__init__.py", line 92, in _do_exit
raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError
It's not clear to me from the logs if a specific device is causing this, or how to trace this back to a specific device or command. If I'm reading correctly at https://docs.python.org/3/library/asyncio-dev.html#detect-never-retrieved-exceptions, it seems like there's an underlying exception that's not being awaited on. What I'm not clear about is if this should be done in this library or somewhere further up the stack. Any suggestions on how to debug this further?
🤦 this is what I get for setting up a bot account at the same time.
I don't think this is related, but just in case, I'm also getting the following warnings:
Logger: zigpy_znp.api
Source: /usr/local/lib/python3.8/site-packages/zigpy_znp/api.py:395
First occurred: 5:29:42 PM (5 occurrences)
Last logged: 5:30:58 PM
Received an unhandled command: ZDO.ConcentratorInd.Callback(NWK=0x64B4, IEEE=5c:02:72:ff:fe:ce:11:e4, PktCost=1)
Received an unhandled command: ZDO.ConcentratorInd.Callback(NWK=0xE797, IEEE=5c:02:72:ff:fe:ce:0c:cb, PktCost=1)
Received an unhandled command: AF.DataConfirm.Callback(Status=<Status.SUCCESS: 0>, Endpoint=1, TSN=227)
Both devices are Jasco light switches, often branded as GE or Enbrighten:
{
"node_descriptor": "NodeDescriptor(byte1=1, byte2=64, mac_capability_flags=142, manufacturer_code=4388, maximum_buffer_size=82, maximum_incoming_transfer_size=255, server_mask=11264, maximum_outgoing_transfer_size=255, descriptor_capability_field=0, *allocate_address=True, *complex_descriptor_available=False, *is_alternate_pan_coordinator=False, *is_coordinator=False, *is_end_device=False, *is_full_function_device=True, *is_mains_powered=True, *is_receiver_on_when_idle=True, *is_router=True, *is_security_capable=False, *is_valid=True, *logical_type=<LogicalType.Router: 1>, *user_descriptor_available=False)",
"endpoints": {
"1": {
"profile_id": 260,
"device_type": "0x0100",
"in_clusters": [
"0x0000",
"0x0003",
"0x0004",
"0x0005",
"0x0006",
"0x0b05"
],
"out_clusters": [
"0x000a",
"0x0019"
]
},
"2": {
"profile_id": 260,
"device_type": "0x0103",
"in_clusters": [
"0x0000",
"0x0003",
"0x0b05"
],
"out_clusters": [
"0x0003",
"0x0006"
]
},
"242": {
"profile_id": 41440,
"device_type": "0x0061",
"in_clusters": [],
"out_clusters": [
"0x0021"
]
}
},
"manufacturer": "Jasco Products",
"model": "43076",
"class": "zigpy.device.Device"
}
{
"node_descriptor": "NodeDescriptor(byte1=1, byte2=64, mac_capability_flags=142, manufacturer_code=4388, maximum_buffer_size=82, maximum_incoming_transfer_size=255, server_mask=11264, maximum_outgoing_transfer_size=255, descriptor_capability_field=0, *allocate_address=True, *complex_descriptor_available=False, *is_alternate_pan_coordinator=False, *is_coordinator=False, *is_end_device=False, *is_full_function_device=True, *is_mains_powered=True, *is_receiver_on_when_idle=True, *is_router=True, *is_security_capable=False, *is_valid=True, *logical_type=<LogicalType.Router: 1>, *user_descriptor_available=False)",
"endpoints": {
"1": {
"profile_id": 260,
"device_type": "0x0101",
"in_clusters": [
"0x0000",
"0x0003",
"0x0004",
"0x0005",
"0x0006",
"0x0008",
"0x0b05"
],
"out_clusters": [
"0x000a",
"0x0019"
]
},
"2": {
"profile_id": 260,
"device_type": "0x0104",
"in_clusters": [
"0x0000",
"0x0003",
"0x0b05"
],
"out_clusters": [
"0x0003",
"0x0006",
"0x0008"
]
},
"242": {
"profile_id": 41440,
"device_type": "0x0061",
"in_clusters": [],
"out_clusters": [
"0x0021"
]
}
},
"manufacturer": "Jasco Products",
"model": "43080",
"class": "zigpy.device.Device"
}
ZHA debug logging will give you all the necessary context to track this down.
Thanks, I was missing a few components from that list.
Based on https://github.com/zigpy/zigpy-znp/blob/dev/zigpy_znp/config.py#L69, I'm looking at the 30 seconds of logs from before the exception is thrown. There's several notices along the lines of "Extending timeout... for request":
https://gist.github.com/deviantintegral/2ce4aecf57efdff4758e687ffe5f7489#file-74-task-exception-log-L43
All of those devices are Sengled Z01-A19NAE26 bulbs.
Looking at https://github.com/zigpy/zigpy/blob/78cefbca987a7188b556e189e58fb6348a67a4ab/zigpy/device.py#L265-L266, it seems like that message is logged for all requests, and may not indicate an issue.
Thanks. Are there any log lines around 10:39:57? The snippet you posted only covers 28 seconds and the original request might be a little before that.
The prior log line was 10:39:48, so beyond the window:
2021-06-14 10:39:48 DEBUG (MainThread) [zigpy_znp.api] Received command: AF.DataConfirm.Callback(Status =<Status.SUCCESS: 0>, Endpoint=1, TSN=61)
Can you add the following to your logging config to enable very verbose logging for zigpy-znp?
zigpy_znp.logger: DEBUG
If you can get the error to occur again, everything should be in the preceding five minutes of log entries.
The above was issue was fixed (🎉 !), as have other fixes to the library since I reported this issue. I'm still seeing the above task exception logged, and based on the "Extending Timeout" messages it's still all Sengled bulbs, either E21-N1EA or Z01-A19NAE26. I have a few Sengled outlets and they don't show up. It doesn't look like there's firmware updates available either, looking at the observed firmware listed at https://github.com/dresden-elektronik/deconz-rest-plugin/wiki/OTA-Image-Types---Firmware-versions
In terms of functionality, I'm noticing occasional lag (15-30 seconds) on responding to events on two sets of motion lights. I also have one Zigbee group that has a mix of Sengled and Ikea bulbs. The Ikea bulbs aren't showing up suspiciously in the logs at all.
I'm planning on replacing these bulbs, as I've also had issues with premature failures. They were cheap, but perhaps too cheap! So, it's entirely possible this is due to some issue with the bulbs themselves.
Full log with the preceding 5 minutes is at https://gist.github.com/deviantintegral/cf6849d59fb8b0d0e0c39ba4355d1eeb
2021-12-19 13:22:14 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved: File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/usr/src/homeassistant/homeassistant/__main__.py", line 331, in <module>
sys.exit(main())
File "/usr/src/homeassistant/homeassistant/__main__.py", line 318, in main
exit_code = runner.run(runtime_conf)
File "/usr/src/homeassistant/homeassistant/runner.py", line 121, in run
return loop.run_until_complete(setup_and_run_hass(runtime_config))
File "/usr/local/lib/python3.9/asyncio/base_events.py", line 629, in run_until_complete
self.run_forever()
File "/usr/local/lib/python3.9/asyncio/base_events.py", line 596, in run_forever
self._run_once()
File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1882, in _run_once
handle._run()
File "/usr/local/lib/python3.9/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "/usr/src/homeassistant/homeassistant/util/logging.py", line 140, in async_wrapper
await async_func(*args)
File "/usr/src/homeassistant/homeassistant/components/zha/light.py", line 518, in _maybe_force_refresh
await self.async_get_state()
File "/usr/src/homeassistant/homeassistant/components/zha/light.py", line 460, in async_get_state
state = await self._on_off_channel.get_attribute_value(
File "/usr/src/homeassistant/homeassistant/components/zha/core/channels/base.py", line 377, in get_attribute_value
result = await safe_read(
File "/usr/src/homeassistant/homeassistant/components/zha/core/helpers.py", line 65, in safe_read
result, _ = await cluster.read_attributes(
File "/usr/local/lib/python3.9/site-packages/zigpy/zcl/__init__.py", line 297, in read_attributes
result = await self.read_attributes_raw(to_read, manufacturer=manufacturer)
File "/usr/local/lib/python3.9/site-packages/zigpy/device.py", line 265, in request
result, msg = await self._application.request(
File "/usr/local/lib/python3.9/site-packages/zigpy_znp/zigbee/application.py", line 453, in request
return await self._send_request(
File "/usr/local/lib/python3.9/site-packages/zigpy_znp/zigbee/application.py", line 1483, in _send_request
response = await self._send_request_raw(
File "/usr/local/lib/python3.9/site-packages/zigpy_znp/zigbee/application.py", line 1368, in _send_request_raw
response = await asyncio.shield(
File "/usr/local/lib/python3.9/asyncio/tasks.py", line 881, in shield
inner = ensure_future(arg, loop=loop)
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/zigpy_znp/api.py", line 826, in request
response = await response_future
asyncio.exceptions.CancelledError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/zigpy_znp/api.py", line 865, in request_callback_rsp
await self.request(request, **response_params)
File "/usr/local/lib/python3.9/site-packages/zigpy_znp/api.py", line 826, in request
response = await response_future
File "/usr/local/lib/python3.9/site-packages/async_timeout/__init__.py", line 128, in __aexit__
self._do_exit(exc_type)
File "/usr/local/lib/python3.9/site-packages/async_timeout/__init__.py", line 203, in _do_exit
raise asyncio.TimeoutError
asyncio.exceptions.TimeoutError
@deviantintegral I know this thread is old, but did you ever find out more information about the Sengled bulb issues?
I've got a bunch (about 40) Sengled bulbs and they've got me pulling my hair out. I've done every possible troubleshooting step. Ensured the zigbee channel has no overlapping wifi. Rebuilt the network several times. Tried different router devices. Even tried two different coordinators (Sonoff ZBDongle-P and Sonoff ZBDongle-E) that use different chipsets with both ZHA (zigpy) and Zigbee2MQTT (zigbee_herdsman). The Sengled bulbs still drop off the network, fail to respond to commands, etc.
I'm really hoping I can find a solution somehow but information is sparse. I'd love to avoid replacing all these bulbs.
I didn't. However, I've since switched to zigbee2mqtt and the bulbs have the same problem. I'd guess one becomes unresponsive without a power reset every 4-6 weeks. Something notable is that they will usually respond to group commands, but not individual device commands. I've slowly been replacing my bulbs with Hue bulbs. They're more expensive, but are so much more reliable.
Honestly, I think it's just a bug in their firmware. I just set up a zigbee packet sniffer, so the next time one goes offline I'll see if that shows anything useful.
@puddly any development on this bug?
I'm getting it for more than just Sengled bulbs, coupled with other errors:
Logger: homeassistant
Source: /usr/src/homeassistant/homeassistant/runner.py:145
First occurred: 10:22:27 PM (41 occurrences)
Last logged: 11:51:58 PM
Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/zigpy_znp/api.py", line 1067, in request_callback_rsp
await self.request(request, timeout=timeout, **response_params)
File "/usr/local/lib/python3.11/site-packages/zigpy_znp/api.py", line 1021, in request
self._uart.send(frame)
^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'send'
Also:
Logger: homeassistant.helpers.entity
Source: helpers/entity.py:696
First occurred: 10:22:27 PM (15 occurrences)
Last logged: 11:51:58 PM
Update for sensor.console_power fails
Update for sensor.china_cabinet_humidifier_power fails
Update for sensor.livingroom_humidifier_power fails
Update for sensor.organ_lamp_power fails
Update for sensor.kitchen_speakers_power fails
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 696, in async_update_ha_state
await self.async_device_update()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 957, in async_device_update
await self.async_update()
File "/usr/src/homeassistant/homeassistant/components/zha/sensor.py", line 315, in async_update
await super().async_update()
File "/usr/src/homeassistant/homeassistant/components/zha/entity.py", line 229, in async_update
await asyncio.gather(*tasks)
File "/usr/src/homeassistant/homeassistant/components/zha/core/cluster_handlers/homeautomation.py", line 103, in async_update
result = await self.get_attributes(attrs, from_cache=False, only_cache=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/zha/core/cluster_handlers/__init__.py", line 491, in _get_attributes
read, _ = await self.cluster.read_attributes(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/zigpy/zcl/__init__.py", line 524, in read_attributes
result = await self.read_attributes_raw(to_read, manufacturer=manufacturer)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/zigpy/zcl/__init__.py", line 377, in request
return await self._endpoint.request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/zigpy/endpoint.py", line 253, in request
return await self.device.request(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/zigpy/device.py", line 297, in request
await self._application.request(
File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 768, in request
await self.send_packet(
File "/usr/local/lib/python3.11/site-packages/zigpy_znp/zigbee/application.py", line 1061, in send_packet
await self._send_request_raw(
File "/usr/local/lib/python3.11/site-packages/zigpy_znp/zigbee/application.py", line 952, in _send_request_raw
response = await asyncio.shield(
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/zigpy_znp/api.py", line 1067, in request_callback_rsp
await self.request(request, timeout=timeout, **response_params)
File "/usr/local/lib/python3.11/site-packages/zigpy_znp/api.py", line 1021, in request
self._uart.send(frame)
^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'send'
Also:
Logger: zigpy.zcl
Source: runner.py:188
First occurred: 10:22:27 PM (18 occurrences)
Last logged: 11:51:58 PM
[0x7BFC:1:0x0300] Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/zigpy/zcl/__init__.py", line 411, in reply return await self._endpoint.reply( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy/endpoint.py", line 278, in reply return await self.device.reply( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy/device.py", line 469, in reply return await self.request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy/device.py", line 297, in request await self._application.request( File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 768, in request await self.send_packet( File "/usr/local/lib/python3.11/site-packages/zigpy_znp/zigbee/application.py", line 1061, in send_packet await self._send_request_raw( File "/usr/local/lib/python3.11/site-packages/zigpy_znp/zigbee/application.py", line 952, in _send_request_raw response = await asyncio.shield( ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy_znp/api.py", line 1067, in request_callback_rsp await self.request(request, timeout=timeout, **response_params) File "/usr/local/lib/python3.11/site-packages/zigpy_znp/api.py", line 1021, in request self._uart.send(frame) ^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'send'
[0x7BFC:1:0x0008] Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/zigpy/zcl/__init__.py", line 411, in reply return await self._endpoint.reply( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy/endpoint.py", line 278, in reply return await self.device.reply( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy/device.py", line 469, in reply return await self.request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy/device.py", line 297, in request await self._application.request( File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 768, in request await self.send_packet( File "/usr/local/lib/python3.11/site-packages/zigpy_znp/zigbee/application.py", line 1061, in send_packet await self._send_request_raw( File "/usr/local/lib/python3.11/site-packages/zigpy_znp/zigbee/application.py", line 952, in _send_request_raw response = await asyncio.shield( ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy_znp/api.py", line 1067, in request_callback_rsp await self.request(request, timeout=timeout, **response_params) File "/usr/local/lib/python3.11/site-packages/zigpy_znp/api.py", line 1021, in request self._uart.send(frame) ^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'send'
[0xF39B:1:0xfc7d] Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/zigpy/zcl/__init__.py", line 411, in reply return await self._endpoint.reply( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy/endpoint.py", line 278, in reply return await self.device.reply( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy/device.py", line 469, in reply return await self.request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy/device.py", line 297, in request await self._application.request( File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 768, in request await self.send_packet( File "/usr/local/lib/python3.11/site-packages/zigpy_znp/zigbee/application.py", line 1061, in send_packet await self._send_request_raw( File "/usr/local/lib/python3.11/site-packages/zigpy_znp/zigbee/application.py", line 952, in _send_request_raw response = await asyncio.shield( ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy_znp/api.py", line 1067, in request_callback_rsp await self.request(request, timeout=timeout, **response_params) File "/usr/local/lib/python3.11/site-packages/zigpy_znp/api.py", line 1021, in request self._uart.send(frame) ^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'send'
[0x2DD0:1:0x0019] Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/zigpy/zcl/clusters/general.py", line 2191, in _handle_cluster_request await self._handle_image_block(*args, tsn=tsn, model=self.endpoint.model) File "/usr/local/lib/python3.11/site-packages/zigpy/zcl/clusters/general.py", line 2312, in _handle_image_block await self.image_block_response( File "/usr/local/lib/python3.11/site-packages/zigpy/zcl/__init__.py", line 411, in reply return await self._endpoint.reply( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy/endpoint.py", line 278, in reply return await self.device.reply( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy/device.py", line 469, in reply return await self.request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy/device.py", line 297, in request await self._application.request( File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 768, in request await self.send_packet( File "/usr/local/lib/python3.11/site-packages/zigpy_znp/zigbee/application.py", line 1061, in send_packet await self._send_request_raw( File "/usr/local/lib/python3.11/site-packages/zigpy_znp/zigbee/application.py", line 952, in _send_request_raw response = await asyncio.shield( ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy_znp/api.py", line 1067, in request_callback_rsp await self.request(request, timeout=timeout, **response_params) File "/usr/local/lib/python3.11/site-packages/zigpy_znp/api.py", line 1021, in request self._uart.send(frame) ^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'send'
[0x51CC:1:0x0019] Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/zigpy/zcl/clusters/general.py", line 2191, in _handle_cluster_request await self._handle_image_block(*args, tsn=tsn, model=self.endpoint.model) File "/usr/local/lib/python3.11/site-packages/zigpy/zcl/clusters/general.py", line 2312, in _handle_image_block await self.image_block_response( File "/usr/local/lib/python3.11/site-packages/zigpy/zcl/__init__.py", line 411, in reply return await self._endpoint.reply( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy/endpoint.py", line 278, in reply return await self.device.reply( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy/device.py", line 469, in reply return await self.request( ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy/device.py", line 297, in request await self._application.request( File "/usr/local/lib/python3.11/site-packages/zigpy/application.py", line 768, in request await self.send_packet( File "/usr/local/lib/python3.11/site-packages/zigpy_znp/zigbee/application.py", line 1061, in send_packet await self._send_request_raw( File "/usr/local/lib/python3.11/site-packages/zigpy_znp/zigbee/application.py", line 952, in _send_request_raw response = await asyncio.shield( ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/zigpy_znp/api.py", line 1067, in request_callback_rsp await self.request(request, timeout=timeout, **response_params) File "/usr/local/lib/python3.11/site-packages/zigpy_znp/api.py", line 1021, in request self._uart.send(frame) ^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'send'
Any advice?
@kylepyke your problem is unrelated and happens when the coordinator is disconnected, it's not really something that can be fixed.