pyscript Detected blocking call

Today I got the following warning:

2024-07-15 08:06:25.171 WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to scandir with args (b'/config/pyscript',) inside the event loop by custom integration 'pyscript' at custom_components/pyscript/init.py, line 335: observer.start() (offender: , line 366: ?), please create a bug report at https://github.com/custom-components/pyscript/issues For developers, please see https://developers.home-assistant.io/docs/asyncio_blocking_operations/#scandir Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "/usr/src/homeassistant/homeassistant/main.py", line 223, in sys.exit(main()) File "/usr/src/homeassistant/homeassistant/main.py", line 209, in main exit_code = runner.run(runtime_conf) File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run return loop.run_until_complete(setup_and_run_hass(runtime_config)) File "/usr/local/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete self.run_forever() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 641, in run_forever self._run_once() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1990, in _run_once handle._run() File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) File "/config/custom_components/pyscript/init.py", line 335, in hass_started observer.start()

Jul 15 '24 08:07 Teemops2806

Bug Report: Blocking Call Detected in Pyscript Integration

Description: A blocking call to os.walk() and os.scandir() has been detected inside the event loop by the custom integration pyscript. This issue can affect the performance and responsiveness of Home Assistant.

Logger: homeassistant.util.loop

Source: util/loop.py:136

First Occurrence: 17:57:42

Last Occurrence: 17:57:42

Details: Detected blocking call to walk with args (b'/config/pyscript',) inside the event loop by custom integration 'pyscript' at custom_components/pyscript/__init__.py, line 335: observer.start() (offender: /usr/local/lib/python3.12/site-packages/watchdog/observers/inotify_c.py, line 394: for root, dirnames, _ in os.walk(path):), please create a bug report at https://github.com/custom-components/pyscript/issues For developers, please see https://developers.home-assistant.io/docs/asyncio_blocking_operations/#walk Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module> sys.exit(main()) File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main exit_code = runner.run(runtime_conf) File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run return loop.run_until_complete(setup_and_run_hass(runtime_config)) File "/usr/local/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete self.run_forever() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 641, in run_forever self._run_once() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1990, in _run_once handle._run() File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) File "/usr/src/homeassistant/homeassistant/runner.py", line 166, in setup_and_run_hass return await hass.async_run() File "/usr/src/homeassistant/homeassistant/core.py", line 523, in async_run await self.async_start() File "/usr/src/homeassistant/homeassistant/core.py", line 576, in async_start self.bus.async_fire_internal(EVENT_HOMEASSISTANT_STARTED) File "/usr/src/homeassistant/homeassistant/core.py", line 1559, in async_fire_internal self._hass.async_run_hass_job(job, event) File "/usr/src/homeassistant/homeassistant/core.py", line 941, in async_run_hass_job return self._async_add_hass_job(hassjob, *args, background=background) File "/usr/src/homeassistant/homeassistant/core.py", line 756, in _async_add_hass_job task = create_eager_task( File "/usr/src/homeassistant/homeassistant/util/async_.py", line 37, in create_eager_task return Task(coro, loop=loop, name=name, eager_start=True) File "/config/custom_components/pyscript/__init__.py", line 335, in hass_started observer.start() Detected blocking call to scandir with args (b'/config/pyscript',) inside the event loop by custom integration 'pyscript' at custom_components/pyscript/__init__.py, line 335: observer.start() (offender: <frozen os>, line 366: ?), please create a bug report at https://github.com/custom-components/pyscript/issues For developers, please see https://developers.home-assistant.io/docs/asyncio_blocking_operations/#scandir Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module> sys.exit(main()) File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main exit_code = runner.run(runtime_conf) File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run return loop.run_until_complete(setup_and_run_hass(runtime_config)) File "/usr/local/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete self.run_forever() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 641, in run_forever self._run_once() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1990, in _run_once handle._run() File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) File "/usr/src/homeassistant/homeassistant/runner.py", line 166, in setup_and_run_hass return await hass.async_run() File "/usr/src/homeassistant/homeassistant/core.py", line 523, in async_run await self.async_start() File "/usr/src/homeassistant/homeassistant/core.py", line 576, in async_start self.bus.async_fire_internal(EVENT_HOMEASSISTANT_STARTED) File "/usr/src/homeassistant/homeassistant/core.py", line 1559, in async_fire_internal self._hass.async_run_hass_job(job, event) File "/usr/src/homeassistant/homeassistant/core.py", line 941, in async_run_hass_job return self._async_add_hass_job(hassjob, *args, background=background) File "/usr/src/homeassistant/homeassistant/core.py", line 756, in _async_add_hass_job task = create_eager_task( File "/usr/src/homeassistant/homeassistant/util/async_.py", line 37, in create_eager_task return Task(coro, loop=loop, name=name, eager_start=True) File "/config/custom_components/pyscript/__init__.py", line 335, in hass_started observer.start() After Update 2024.6.x to 2024.7.3

Jul 23 '24 16:07 Just-Zuul

I have the same issue. This is my code that causes that blocking call warning:

        for t in dir_types:
            if t in source:
                inc = source.split(t + ' ')
                path = '/config/' + inc[1].replace('./','')
                with os.scandir(path) as it:
                    for entry in it:
                        if not entry.name.startswith('.') and entry.is_file():
                            includes.append(path + '/' + entry.name)
                return

Nov 27 '24 14:11 erkr

This is because you are making file system calls (eg, os.scandir), which can block. Pyscript runs in the main event loop so you cannot call functions that block. There are two solutions:

use async versions of file system calls, eg, aiopath.
use task.executor to run the blocking code outside the main event loop. See the documentation.

Nov 27 '24 18:11 craigbarratt

Hi there,as I am not a dev, I don't understand a single word 😁I hope, some skilled dev reads this and can help/fix this.Using the onboard backup function should work, I guess.Thanks for checkimg this topic.Best regardsFreundliche Grüße, FrankAm 27.11.2024 um 19:55 schrieb Craig Barratt @.***>: This is because you are making file system calls (eg, os.scandir), which can block. Pyscript runs in the main event loop so you cannot call functions that block. There are two solutions:

use async versions of file system calls, eg, aiopath. use task.executor to run the blocking code outside the main event loop. See the documentation.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

Nov 27 '24 20:11 Teemops2806

Thanks, I was under the impression that pyscripts were running asynchronous by design. Will look into your leads and learn ☺️

Nov 28 '24 09:11 erkr

@craigbarratt Thanks for the leads. I just needed to change one line to fix it 🙏:

with os.scandir(path) as it:

Into

with task.executor(os.scandir,path) as it:

Dec 01 '24 18:12 erkr

As of HA Core 2024.12.3 and hacs-pyscript 1.6.1:

This is not a bug but how HomeAssistant and Pyscript behave. The error message can also look like caught blocking call to putrequest if the blocking operation caught is some http request done by a library.

I found a way how to get around, but it requires making a non-async python module in /config/pyscript_modules (note that /config and /homeassistant are the same) like in the docs and at least in my case, it is heavily cached and at least in my case, I found no way how to refresh any changes in code in the pyscript_modules directory once it is detected and cached, other than restart. Which is really annoying, so I would love to hear about some better way.

Anyway, how to deal with this:

Create /config/pyscript_modules for synchronous/standard methods, including __init__.py, like mentioned in the docs.

Put there file with the blocking code, e.g. in my case, I made /config/pyscript_modules/blocking_code.py with this content:

 import requests

 def requests_test(log):
     response = requests.get(url = "https://www.google.com")
     msg = f"Status code: {response.status_code}\nResponse text (snippet): {response.text[:100]}"
     log("it works")
     return msg

In /config/pyscript/my_service.py I have this code:

 import sys
 if "/config/pyscript_modules" not in sys.path:
     sys.path.append("/config/pyscript_modules")
 import blocking_code as bs # located in pyscript_modules

 @service
 async def hello_world_requests(action=None, id=None):
     # log.warning passed as a callback as an example parameter
     result = task.executor(bs.requests_test, log.warning) 
     log.error(result)

And when this service is called, it does the http request correctly and logs one error and one warning with expected messages. (because I'm too lazy to change log level to debug)

Dec 15 '24 16:12 jtulak

Hello,maybe not "a real bug" but if you use pyscript and therefore all your backups are somehow corrupt and cannot be used, this is something quiet annoying.Would be nice, if somebody finds a solution for this.Best regards, FrankAm 15.12.2024 um 17:56 schrieb Jan Ťulák @.***>: As of HA Core 2024.12.3 and hacs-pyscript 1.6.1: This is not a bug but how HomeAssistant and Pyscript behave. The error message can also look like caught blocking call to putrequest if the blocking operation caught is some http request done by a library. I found a way how to get around, but it requires making a non-async python module in /config/pyscript_modules (note that /config and /homeassistant are the same) like in the docs and at least in my case, it is heavily cached and at least in my case, I found no way how to refresh any changes in code in the pyscript_modules directory once it is detected and cached, other than restart. Which is really annoying, so I would love to hear about some better way. Anyway, how to deal with this:

Create /config/pyscript_modules for synchronous/standard methods, including init.py, like mentioned in the docs.

Put there file with the blocking code, e.g. in my case, I made /config/pyscript_modules/blocking_code.py with this content: import requests

def requests_test(log): response = requests.get(url = "https://www.google.com") msg = f"Status code: {response.status_code}\nResponse text (snippet): {response.text[:100]}" log("it works") return msg

In /config/pyscript/my_service.py I have this code: import sys if "/config/pyscript_modules" not in sys.path: sys.path.append("/config/pyscript_modules") import blocking_code as bs # located in pyscript_modules

@service async def hello_world_requests(action=None, id=None): # log.warning passed as a callback as an example parameter result = task.executor(bs.requests_test, log.warning) log.error(result)

And when this service is called, it does the http request correctly and logs one error and one warning with expected messages. (because I'm too lazy to change log level to debug) image.png (view on web)

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

Dec 15 '24 17:12 Teemops2806

@Teemops2806 Wait, so you can't restore a backup because of something like this? Or is it just some script you are running using pyscript that stopped working?

Dec 15 '24 17:12 jtulak