Shelly: aioshelly (Concurrent call to receive() is not allowed")

[martinarva]

The problem

Shelly integration has crached my HA since yesterday. I guess it's related to 2023.12 update which happened also yesterday. Happened once yesterday and now this morning again. Had to restart the HA docker to be able to access HA UI again.

HA just hangs and UI is not available. Last log items are these (like 10 of them):

File "/usr/local/lib/python3.11/site-packages/aiohttp/client_ws.py", line 232, in receive raise RuntimeError("Concurrent call to receive() is not allowed") RuntimeError: Concurrent call to receive() is not allowed 2023-12-08 07:12:48.593 WARNING (MainThread) [aioshelly.rpc_device.wsrpc] Response for an unknown request id: 755 2023-12-08 07:12:48.593 ERROR (MainThread) [aioshelly.rpc_device.wsrpc] Unexpected error while receiving message Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/aioshelly/rpc_device/wsrpc.py", line 328, in _rx_msgs msg = await self._client.receive()

What version of Home Assistant Core has the issue?

core-2023.12.0

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Core

Integration causing the issue

Shelly

Link to integration documentation on our website

https://www.home-assistant.io/integrations/shelly/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[home-assistant[bot]]

Hey there @balloob, @bieniu, @thecode, @chemelli74, @bdraco, mind taking a look at this issue as it has been labeled with an integration (shelly) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of shelly can trigger bot actions by commenting:

• @home-assistant close Closes the issue.
• @home-assistant rename Awesome new title Renames the issue.
• @home-assistant reopen Reopen the issue.
• @home-assistant unassign shelly Removes the current integration label and assignees on the issue, add the integration domain after the command.
• @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
• @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

---

shelly documentation shelly source _{^{(message by IssueLinks)}}

[chemelli74]

Hi, could you please post:

• Information screen of your HA installation
• diagnostic of the device
• full debug log

[branbon]

I also am experiencing this, and it also causes enough log-spam that my home-assistant client becomes unresponsive & over the span of ~18 hours or so has logged ~60GB worth of those repeating messages. I also was not experiencing this prior to 2023.12. I am running via Docker on a Radxa Rock 5 Model B.

I can try to provide additional details if needed:

2023-12-09 18:52:55.529 ERROR (MainThread) [aioshelly.rpc_device.wsrpc] 192.168.1.220: Pong not received, device is likely unresponsive; disconnecting
2023-12-09 18:53:33.040 ERROR (MainThread) [aioshelly.rpc_device.wsrpc] Unexpected error while receiving message
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aioshelly/rpc_device/wsrpc.py", line 328, in _rx_msgs
    msg = await self._client.receive()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_ws.py", line 232, in receive
    raise RuntimeError("Concurrent call to receive() is not allowed")
RuntimeError: Concurrent call to receive() is not allowed
2023-12-09 18:53:33.047 WARNING (MainThread) [aioshelly.rpc_device.wsrpc] Response for an unknown request id: 231
2023-12-09 18:53:33.050 ERROR (MainThread) [aioshelly.rpc_device.wsrpc] Unexpected error while receiving message
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aioshelly/rpc_device/wsrpc.py", line 328, in _rx_msgs
    msg = await self._client.receive()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_ws.py", line 232, in receive
    raise RuntimeError("Concurrent call to receive() is not allowed")
RuntimeError: Concurrent call to receive() is not allowed

I have also enabled debug logging on the integration and will report any additional details that might show up there.

[thecode]

I have suspect it is related to https://github.com/home-assistant-libs/aioshelly/pull/426 becasue it has a slight change how we handle futures, @bdraco what do you think?

[chemelli74]

@brandond please share all the requested information

[branbon]

• Information screen of your HA installation (pulled via settings -> system -> repairs, 3-dot menu in corner)

  System Information

  version	core-2023.12.1
  installation_type	Home Assistant Container
  dev	false
  hassio	false
  docker	true
  user	root
  virtualenv	false
  python_version	3.11.6
  os_name	Linux
  os_version	5.10.160-legacy-rk35xx
  arch	aarch64
  timezone	America/New_York
  config_dir	/config

  Home Assistant Cloud

  logged_in	false
  can_reach_cert_server	ok
  can_reach_cloud_auth	ok
  can_reach_cloud	ok

  Dashboards

  dashboards	3
  resources	0
  views	1
  mode	storage

  Recorder

  oldest_recorder_run	December 1, 2023 at 6:25 PM
  current_recorder_run	December 11, 2023 at 10:17 AM
  estimated_db_size	246.84 MiB
  database_engine	sqlite
  database_version	3.41.2

• diagnostic of the device (sensitive info removed) diagnostics-shelly-plugus-data-device-with-issue.txt

• full debug log (15min leading up to event, sensitive info removed) shelly-logs.log

@chemelli74 I assume you meant to tag me, and I was waiting until I was able to trigger the issue with debug logging enabled as the issue is sporadic and tends to only happen at night when I am asleep.

[brandond]

Wrong @ there, I don't think you wanted me.

[stuckinger]

Same Here. I could got it working again with a rebuild of the container after the upgrade. I do not know which docker version was running before upgrade. After rebuild the high load and log flooding is gone! Shelly integration also works.

Current HA info:

System Information

version	core-2023.12.1
installation_type	Home Assistant Container
dev	false
hassio	false
docker	true
user	root
virtualenv	false
python_version	3.11.6
os_name	Linux
os_version	6.1.0-15-amd64
arch	x86_64
timezone	Europe/Berlin
config_dir	/config

Home Assistant Cloud

logged_in	false
can_reach_cert_server	ok
can_reach_cloud_auth	ok
can_reach_cloud	ok

Dashboards

dashboards	12
resources	1
views	14
mode	storage

Recorder

oldest_recorder_run	2. Dezember 2023 um 21:38
current_recorder_run	12. Dezember 2023 um 07:31
estimated_db_size	986.74 MiB
database_engine	sqlite
database_version	3.41.2

Maybe it helps?

[martinarva]

Same Here. I could got it working again with a rebuild of the container after the upgrade. I do not know which docker version was running before upgrade. After rebuild the high load and log flooding is gone! Shelly integration also works.

Current HA info:

System Information

version core-2023.12.1 installation_type Home Assistant Container dev false hassio false docker true user root virtualenv false python_version 3.11.6 os_name Linux os_version 6.1.0-15-amd64 arch x86_64 timezone Europe/Berlin config_dir /config Home Assistant Cloud logged_in false can_reach_cert_server ok can_reach_cloud_auth ok can_reach_cloud ok Dashboards dashboards 12 resources 1 views 14 mode storage Recorder oldest_recorder_run 2. Dezember 2023 um 21:38 current_recorder_run 12. Dezember 2023 um 07:31 estimated_db_size 986.74 MiB database_engine sqlite database_version 3.41.2 Maybe it helps?

Did the same. At the moment looks stable

[christofnolden]

Same here... Spammed the log file until the disk was full...

File "/usr/local/lib/python3.11/site-packages/aiohttp/client_ws.py", line 232, in receive raise RuntimeError("Concurrent call to receive() is not allowed") RuntimeError: Concurrent call to receive() is not allowed 2023-12-12 03:39:52.414 WARNING (MainThread) [aioshelly.rpc_device.wsrpc] Response for an unknown request id: 13 2023-12-12 03:39:52.414 ERROR (MainThread) [aioshelly.rpc_device.wsrpc] Unexpected error while receiving message Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/aioshelly/rpc_device/wsrpc.py", line 328, in _rx_msgs msg = await self._client.receive() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

EDIT: The recreation / rebuilt of the container fixed the problem.

[branbon]

Since encountering this last, I have:

• power cycled the device
• updated to the latest beta firmware on my shelly plug
• relocated the device causing the issue to be closer to the nearest wireless access point
• updated to the latest home-assistant container version (2023.12.3)

Since doing all that, the container has been up and running without having this issue thrown. I believe I encountered it today (home-assistant became unresponsive & CPU usage became pinned to 15%-20%), but have also added this to my configuration.yaml to avoid running out of disk space (so will no longer see the main error logs I included previously):

logger:
  default: warning
  filters:
    # filter for aioshelly.rpc_device.wsrpc errors
    aioshelly.rpc_device.wsrpc:
      - "^Unexpected error while receiving message.*$"
      - "^Response for an unknown request id.*$"

[oxigen-dev]

Same happened to me, today for the 2nd time ever (1st time was some days ago). With restarting the docker container I can get back to the UI.

[mf76130]

had the same, then i saw 1 dead shelly device in my shelly setup (not in home assistant), deleted that one from the shelly app and since that time silence.

[mxilievski]

This one is bugging me out for the past one week. Last time occurred when my network equipment started to update at 3 AM, dropping around 20 Shelly devices out for about 10 minutes.

[chemelli74]

@all, which other devices you have, if any, in your HA installation that doesn't suffer this issue ?

[mxilievski]

@ALL, which other devices you have, if any, in your HA installation that doesn't suffer this issue ?

I have a mix of Shelly 1PM, 2PM, 1 Plus, RGBW2, 2.5.

Looking at the logs, there is nothing obvious about the device that is causing it.

The rest of the log is the just the error repeating itself.

Update: If by any chance this error is related This error is only coming from 2PM, 1 Plus, 1PM devices Gen2. Gen1 devices are not throwing the error above.

[oxigen-dev]

@ALL, which other devices you have, if any, in your HA installation that doesn't suffer this issue ?

I have:

• Shelly Plug S x4
• Shelly Plus 2PM x1
• Shelly Plus 1 x3
• Shelly Door/Window 2 x1
• Shelly RGBW2 x2

I'm also not sure which is causing the issue.

[chemelli74]

Sorry, I reword the question that was not clear enough: what other brand devices a part from Shelly you have in your HA environment ?

[bieniu]

Error seems to be related to websocket https://github.com/aio-libs/aiohttp/issues/999 so I think it's only affects gen2 devices.

Please provide the following information:

• HA installation type
• Shelly Plus/Pro device firmware version

If Pro/Plus devices are using beta firmware, please roll back to the stable version and re-test HA stability.

[mxilievski]

Error seems to be related to websocket aio-libs/aiohttp#999 so I think it's only affects gen2 devices.

Please provide the following information:

• HA installation type
• Shelly Plus/Pro device firmware version

If Pro/Plus devices are using beta firmware, please roll back to the stable version and re-test HA stability.

HA installation type: Docker, latest image 2023.12.3

Shelly Plus/Pro device firmware version: All 2gen devices are on their latest stable firmware 20231106-160328/1.0.8-gdba0ee3

[ruimarinho]

I am also experiencing this problem randomly. CPU usage skyrockets due to log spam. Anything I can do to help troubleshoot this? I am running a mixed environment with Shelly Gen1 and Gen2.

[oxigen-dev]

Error seems to be related to websocket aio-libs/aiohttp#999 so I think it's only affects gen2 devices.

Please provide the following information:

* HA installation type

* Shelly Plus/Pro device firmware version

If Pro/Plus devices are using beta firmware, please roll back to the stable version and re-test HA stability.

HA installation type: Docker. All of my Gen2 Shelly devices are on the latest stable firmware: 20231106-160229/1.0.8-gdba0ee3 and 20231106-160328/1.0.8-gdba0ee3.

[bieniu]

Is anyone having these problems using a different type of installation than "Home Assistant Container"?

[dannytsang]

@bieniu mine is a VM install

[bieniu]

@dannytsang Please attach a diagnostic file.

[ruimarinho]

I'm running via the official docker image in host mode.

[branbon]

Here are the logs from my most recent event, (excluding "^Unexpected error while receiving message.*$" & "^Response for an unknown request id.*$" logs that is):

shelly-logs-2.log

All of my shelly devices are just Shelly Plug Plus US. I have just updated all deices to the latest stable 1.1.0 device firmware that was released today, (was running into this on 1.0.8 and was what prompted me to update to 1.1.0-beta* in an attempt to resolve the issue).

[dersch81]

Hi, i have daily unexpected reboots of HA due to this issue. Started from 2023.12.0 now i'm on 2023.12.3. I'm running HASSOS on Proxmox VM.

I have a lot if Gen2 devices but also Gen1 Shellies. Updated all Gen2 to 1.1.0 today. Except one which has unstable wifi connection. It is still on 1.0.8

That is the last log before the last unexpected reboot

2023-12-21 10:53:22.582 WARNING (MainThread) [aioshelly.rpc_device.wsrpc] Response for an unknown request id: 1880
2023-12-21 10:53:22.582 ERROR (MainThread) [aioshelly.rpc_device.wsrpc] Unexpected error while receiving message
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aioshelly/rpc_device/wsrpc.py", line 328, in _rx_msgs
    msg = await self._client.receive()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_ws.py", line 232, in receive
    raise RuntimeError("Concurrent call to receive() is not allowed")
RuntimeError: Concurrent call to receive() is not allowed
2023-12-21 10:53:22.583 WARNING (MainThread) [aioshelly.rpc_device.wsrpc] Response for an unknown request id: 1880
2023-12-21 10:53:22.583 ERROR (MainThread) [aioshelly.rpc_device.wsrpc] Unexpected error while receiving message
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aioshelly/rpc_device/wsrpc.py", line 328, in _rx_msgs
    msg = await self._client.receive()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_ws.py", line 232, in receive
    raise RuntimeError("Concurrent call to receive() is not allowed")
RuntimeError: Concurrent call to receive() is not allowed
2023-12-21 10:53:22.584 WARNING (MainThread) [aioshelly.rpc_device.wsrpc] Response for an unknown request id: 1880
2023-12-21 10:53:22.584 ERROR (MainThread) [aioshelly.rpc_device.wsrpc] Unexpected error while receiving message
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aioshelly/rpc_device/wsrpc.py", line 328, in _rx_msgs
    msg = await self._client.receive()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_ws.py", line 232, in receive
    raise RuntimeError("Concurrent call to receive() is not allowed")
RuntimeError: Concurrent call to receive() is not allowed
2023-12-21 10:53:22.585 WARNING (MainThread) [aioshelly.rpc_device.wsrpc] Response for an unknown request id: 1880

Is any information still necessary to find the cause?

[dannytsang]

@dannytsang Please attach a diagnostic file.

I haven't had a chance to capture one. The below is an extract from home-assistant.log.1 file where I noticed the issue. It repeats itself so much the file is ~1.9GB :

2023-12-10 15:34:00.827 ERROR (MainThread) [aioshelly.rpc_device.wsrpc] 192.168.20.124: Pong not received, device is likely unresponsive; disconnecting
2023-12-10 15:34:02.085 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/sensor/__init__.py", line 616, in state
    numerical_value = int(value)
                      ^^^^^^^^^^
ValueError: invalid literal for int() with base 10: '"0"4'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/sensor/__init__.py", line 619, in state
    numerical_value = float(value)
                      ^^^^^^^^^^^^
ValueError: could not convert string to float: '"0"4'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/template/coordinator.py", line 92, in _handle_triggered
    self.async_set_updated_data(
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 426, in async_set_updated_data
    self.async_update_listeners()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 182, in async_update_listeners
    update_callback()
  File "/usr/src/homeassistant/homeassistant/components/template/trigger_entity.py", line 55, in _handle_coordinator_update
    self.async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 745, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 845, in _async_write_ha_state
    state, attr = self._async_generate_attributes()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 786, in _async_generate_attributes
    state = self._stringify_state(available)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 751, in _stringify_state
    if (state := self.state) is None:
                 ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/sensor/__init__.py", line 623, in state
    raise ValueError(
ValueError: Sensor sensor.growatt_grid_export_power has device class 'power', state class 'measurement' unit 'kW' and suggested precision 'None' thus indicating it has a numeric value; however, it has the non-numeric value: '"0"4' (<class 'str'>)
2023-12-10 15:34:03.340 ERROR (MainThread) [aioshelly.rpc_device.wsrpc] Unexpected error while receiving message
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aioshelly/rpc_device/wsrpc.py", line 328, in _rx_msgs
    msg = await self._client.receive()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_ws.py", line 232, in receive
    raise RuntimeError("Concurrent call to receive() is not allowed")
RuntimeError: Concurrent call to receive() is not allowed
2023-12-10 15:34:03.341 ERROR (MainThread) [aioshelly.rpc_device.wsrpc] Unexpected error while receiving message
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aioshelly/rpc_device/wsrpc.py", line 328, in _rx_msgs
    msg = await self._client.receive()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_ws.py", line 232, in receive
    raise RuntimeError("Concurrent call to receive() is not allowed")
RuntimeError: Concurrent call to receive() is not allowed

[antsalm]

Similar issue with the 2023.12 version and onwards. I've rolled back to 2023.11.2 and have not experienced it since, nor crashes of my Homeassistant instant ever before, and it has been pretty unchanged for couple of months.

I'm running Home Assistant on Container with ARM64. Shelly 1PMs with 20231219-133934/1.1.0-g34b5d4f Firmware

What I can observe from the logs: Shelly 2gen (Shelly 1PM) device loses connection, it ends up in some kind of an error loop which eventually crashes the HomeAssistant. Sometimes it is able to recover, if its only one device (maybe?) These Shelly devices appearing on logs are connected with D-LINK WiFi extender which has minor unreliability, but has not caused this kind issues.

Currently trying to test with 2023.12.3, and remeditate situation by improving the WIFI-connection and avoiding disconnects.

2023-12-23 09:13:59.308 ERROR (MainThread) [homeassistant.components.shelly] Error fetching Shelly1pm_2 data: Device disconnected: DeviceConnectionError() v2023-12-23 09:14:09.560 ERROR (MainThread) [homeassistant.components.shelly] Error fetching Shelly1pm_4 data: Device disconnected: DeviceConnectionError() 2023-12-23 09:14:38.346 ERROR (MainThread) [homeassistant.components.shelly] Error fetching Shelly1pm_5 data: Device disconnected: DeviceConnectionError() 2023-12-23 09:14:51.369 ERROR (MainThread) [aioshelly.rpc_device.wsrpc] 192.168.107.21: Pong not received, device is likely unresponsive; disconnecting 2023-12-23 09:14:58.567 ERROR (MainThread) [aioshelly.rpc_device.wsrpc] 192.168.107.24: Pong not received, device is likely unresponsive; disconnecting 2023-12-23 09:14:59.460 ERROR (MainThread) [aioshelly.rpc_device.wsrpc] 192.168.107.23: Pong not received, device is likely unresponsive; disconnecting

Multiple of these errors within one second: 2023-12-23 09:15:00.829 ERROR (MainThread) [homeassistant.components.shelly] Unexpected error fetching Shelly1pm_5 data: Already initializing Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 300, in _async_refresh self.data = await self._async_update_data() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/shelly/coordinator.py", line 531, in _async_update_data await self.device.initialize() File "/usr/local/lib/python3.11/site-packages/aioshelly/rpc_device/device.py", line 142, in initialize raise RuntimeError("Already initializing") RuntimeError: Already initializing

It may continue even if devices are connected back: 2023-12-23 09:15:01.294 INFO (MainThread) [aioshelly.rpc_device.wsrpc] Connected to 192.168.107.24 2023-12-23 09:15:01.309 INFO (MainThread) [aioshelly.rpc_device.wsrpc] Connected to 192.168.107.21 2023-12-23 09:15:01.379 INFO (MainThread) [aioshelly.rpc_device.wsrpc] Connected to 192.168.107.23 2023-12-23 09:15:02.659 ERROR (MainThread) [homeassistant.components.shelly] Unexpected error fetching Shelly1pm_2 data: Already initializing Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 300, in _async_refresh self.data = await self._async_update_data() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/shelly/coordinator.py", line 531, in _async_update_data await self.device.initialize() File "/usr/local/lib/python3.11/site-packages/aioshelly/rpc_device/device.py", line 142, in initialize raise RuntimeError("Already initializing") RuntimeError: Already initializing

Eventually it led to these two messages looping which I believe when HA wents unresponsive. Creating roughly 100mb of log in 10minutes.: 2023-12-23 09:15:32.155 WARNING (MainThread) [aioshelly.rpc_device.wsrpc] Response for an unknown request id: 1354 2023-12-23 09:15:32.156 ERROR (MainThread) [aioshelly.rpc_device.wsrpc] Unexpected error while receiving message Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/aioshelly/rpc_device/wsrpc.py", line 328, in _rx_msgs msg = await self._client.receive() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/aiohttp/client_ws.py", line 232, in receive raise RuntimeError("Concurrent call to receive() is not allowed") RuntimeError: Concurrent call to receive() is not allowed