core icon indicating copy to clipboard operation
core copied to clipboard

sia reads codes file in the event loop which does blocking I/O

Open miguelangel-nubla opened this issue 1 year ago • 18 comments

The problem

In HA version 2024.6.0 the logs get flooded with warnings about a blocking call made to sia_codes.json

What version of Home Assistant Core has the issue?

core-2024.6.0

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

SIA Alarm Systems

Link to integration documentation on our website

https://www.home-assistant.io/integrations/sia/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

2024-06-07 11:46:39.282 WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to open with args ('/usr/local/lib/python3.12/site-packages/pysiaalarm/data/sia_codes.json', 'rb') in /usr/local/lib/python3.12/site-packages/pkg_resources/__init__.py, line 1693: return open(self._fn(self.module_path, resource_name), 'rb') inside the event loop
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module>
    sys.exit(main())
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main
    exit_code = runner.run(runtime_conf)
  File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 672, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 639, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1988, in _run_once
    handle._run()
  File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/local/lib/python3.12/site-packages/pysiaalarm/aio/server.py", line 54, in handle_line
    event = self.parse_and_check_event(data)
  File "/usr/local/lib/python3.12/site-packages/pysiaalarm/base_server.py", line 64, in parse_and_check_event
    event = SIAEvent.from_line(line, self.accounts)
  File "/usr/local/lib/python3.12/site-packages/pysiaalarm/event.py", line 161, in from_line
    return SIAEvent(
  File "<string>", line 28, in __init__
  File "/usr/local/lib/python3.12/site-packages/pysiaalarm/event.py", line 266, in __post_init__
    self.set_sia_code()
  File "/usr/local/lib/python3.12/site-packages/pysiaalarm/event.py", line 106, in set_sia_code
    self.sia_code = _load_sia_codes().get(self.code)  # pylint: disable=E1101
  File "/usr/local/lib/python3.12/site-packages/pysiaalarm/data/data.py", line 44, in _load_sia_codes
    data = _load_data(FILE_SIA_CODES)
  File "/usr/local/lib/python3.12/site-packages/pysiaalarm/data/data.py", line 38, in _load_data
    stream = pkg_resources.resource_stream(__name__, file)
  File "/usr/local/lib/python3.12/site-packages/pkg_resources/__init__.py", line 1187, in resource_stream
    return get_provider(package_or_requirement).get_resource_stream(
  File "/usr/local/lib/python3.12/site-packages/pkg_resources/__init__.py", line 1693, in get_resource_stream
    return open(self._fn(self.module_path, resource_name), 'rb')

Additional information

No response

miguelangel-nubla avatar Jun 07 '24 09:06 miguelangel-nubla

Hey there @eavanvalkenburg, mind taking a look at this issue as it has been labeled with an integration (sia) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of sia can trigger bot actions by commenting:

  • @home-assistant close Closes the issue.
  • @home-assistant rename Awesome new title Renames the issue.
  • @home-assistant reopen Reopen the issue.
  • @home-assistant unassign sia Removes the current integration label and assignees on the issue, add the integration domain after the command.
  • @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
  • @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

(message by CodeOwnersMention)


sia documentation sia source (message by IssueLinks)

home-assistant[bot] avatar Jun 07 '24 09:06 home-assistant[bot]

I have the same problem

DaN66O avatar Jun 09 '24 19:06 DaN66O

The same error, any feedback to solve it?

aperezva avatar Jun 18 '24 09:06 aperezva

@eavanvalkenburg do you still maintain this?

miguelangel-nubla avatar Jun 24 '24 12:06 miguelangel-nubla

It seems nobody support a official integration. Really?

aperezva avatar Jun 24 '24 14:06 aperezva

For the most part, integration maintainers are volunteers working on HA in their spare time. It can take while for someone to have free time to pickup issues.

bdraco avatar Jun 24 '24 15:06 bdraco

I know. And it s wonderful this efforts.

The only doubt I have is an official integration has a minimum level of support.

I can understand this situation in a Hacs or no. Official integrations but I don’t know if it s the same situation in a official integrations.

Anybody knows?

And please, thanks a lot to the people who invest 1 minute to make our lives easier

aperezva avatar Jun 24 '24 16:06 aperezva

This problem seems to stem from pysiaalarm using pkg_resources, which then uses blocking io. I guess, as pkg_resources this should be migrated to importlib as the former is deprecated.

However, that might not fix this issue as the maintainer of said package would still have to implement some unblocking io here, using aiofiles or something similar. Not really sure what that would look like, but there's a similar issue fixed here.

frli4797 avatar Jun 26 '24 09:06 frli4797

@eavanvalkenburg do you still maintain this?

DaN66O avatar Sep 09 '24 20:09 DaN66O

For the most part, integration maintainers are volunteers working on HA in their spare time. It can take while for someone to have free time to pickup issues.

@bdraco after 2 months of this issue, it seems nobody is maintaining this integration. Any alternatives to it?

aperezva avatar Sep 13 '24 08:09 aperezva

Same problem here.

Teunhen avatar Sep 26 '24 22:09 Teunhen

I can't recall having this issue before 2024.10. Still no news?

Kolia56 avatar Oct 04 '24 18:10 Kolia56

Another error, Registratore: homeassistant.util.loop Fonte: util/loop.py:77 Prima occorrenza: 17:00:23 (1 occorrenze) Ultima registrazione: 17:00:23

Detected blocking call to open with args ('/usr/local/lib/python3.12/site-packages/pysiaalarm/data/sia_codes.json', 'rb') in /usr/local/lib/python3.12/site-packages/pkg_resources/init.py, line 1758: return open(self._fn(self.module_path, resource_name), 'rb') inside the event loop; This is causing stability issues. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue For developers, please see https://developers.home-assistant.io/docs/asyncio_blocking_operations/#open Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "/usr/src/homeassistant/homeassistant/main.py", line 223, in sys.exit(main()) File "/usr/src/homeassistant/homeassistant/main.py", line 209, in main exit_code = runner.run(runtime_conf) File "/usr/src/homeassistant/homeassistant/runner.py", line 189, in run return loop.run_until_complete(setup_and_run_hass(runtime_config)) File "/usr/local/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete self.run_forever() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 641, in run_forever self._run_once() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1990, in _run_once handle._run() File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) File "/usr/local/lib/python3.12/site-packages/pysiaalarm/aio/server.py", line 54, in handle_line event = self.parse_and_check_event(data) File "/usr/local/lib/python3.12/site-packages/pysiaalarm/base_server.py", line 64, in parse_and_check_event event = SIAEvent.from_line(line, self.accounts) File "/usr/local/lib/python3.12/site-packages/pysiaalarm/event.py", line 161, in from_line return SIAEvent( File "", line 28, in init File "/usr/local/lib/python3.12/site-packages/pysiaalarm/event.py", line 266, in post_init self.set_sia_code() File "/usr/local/lib/python3.12/site-packages/pysiaalarm/event.py", line 106, in set_sia_code self.sia_code = _load_sia_codes().get(self.code) # pylint: disable=E1101 File "/usr/local/lib/python3.12/site-packages/pysiaalarm/data/data.py", line 44, in _load_sia_codes data = _load_data(FILE_SIA_CODES) File "/usr/local/lib/python3.12/site-packages/pysiaalarm/data/data.py", line 38, in _load_data stream = pkg_resources.resource_stream(name, file) File "/usr/local/lib/python3.12/site-packages/pkg_resources/init.py", line 1248, in resource_stream return get_provider(package_or_requirement).get_resource_stream( File "/usr/local/lib/python3.12/site-packages/pkg_resources/init.py", line 1758, in get_resource_stream return open(self._fn(self.module_path, resource_name), 'rb')

Asdrubale1234 avatar Nov 22 '24 16:11 Asdrubale1234

https://github.com/eavanvalkenburg/pysiaalarm/commit/2e2649630323d8999133123bbf23fd613178b615#diff-671d351f7832b66dcd494ce40ed352c6b49a22781ba08d4891670d3891bde599R51

Checkout this fix thanks voskuh

RPJacobs avatar Jan 05 '25 15:01 RPJacobs

Hi, any news about it? is there any alternative fork that is providing a solution? thanks

alexsalvo1 avatar Jan 24 '25 15:01 alexsalvo1

I would like to express my interest in getting this bug fixed. Thanks.

czaftan avatar Mar 07 '25 06:03 czaftan

I'm affected too

What happens when a integration is not maintained ? Why is available ? If someone has a workaround, I'm interested.

Thank you Community.

Ninja-ca avatar Mar 25 '25 16:03 Ninja-ca

WORKAROUND !!!

  1. Use some way to access by SSH to your HA (in my case using "Advanced SSH & Web Terminal" - Disable "Protection Mode")
  2. Access to the HA docker: sudo docker exec -it homeassistant sh
  3. Go to the SIA Protocol Folder: cd /usr/local/lib/python3.13/site-packages/pysiaalarm/data
  4. Add missing SIA Codes: vi sia_codes.json (follow syntaxys from other codes) - Start typping: "insert" button
  5. Save & Exit: "Escape" button + :wq!

After that, your Ajax Systems will be online again. I checked and file "sia_codes.json" is missing a lot of entries.

Ninja-ca avatar Mar 25 '25 17:03 Ninja-ca

so we have 2 solutions above:

ssh root@homeassistant -p 22222      
docker exec -it homeassistant sh
cd /usr/local/lib/python3.13/site-packages/pysiaalarm/data
 vi sia_codes.json

but then it's not clear to me what "missing sia codes" I should add (I believe there are 325 entries in that file already)

apply the changes from here https://github.com/eavanvalkenburg/pysiaalarm/commit/2e2649630323d8999133123bbf23fd613178b615#diff-671d351f7832b66dcd494ce40ed352c6b49a22781ba08d4891670d3891bde599R51

but it's not clear to me how to apply this and how to not have it overwritten at each of the frequent homeassistant updates.

Is anyone more experienced and able to help?

Thanks

fabriba avatar May 13 '25 14:05 fabriba

I´m sorry but if SIA is a oficial integration anyone would solve the problem that appears?

Not all the Home assitant user has knowledge to do this work arrounds.

This error started half of year ago.

aperezva avatar May 13 '25 15:05 aperezva

so we have 2 solutions above:

ssh root@homeassistant -p 22222      
docker exec -it homeassistant sh
cd /usr/local/lib/python3.13/site-packages/pysiaalarm/data
 vi sia_codes.json

but then it's not clear to me what "missing sia codes" I should add (I believe there are 325 entries in that file already)

apply the changes from here eavanvalkenburg/pysiaalarm@2e26496#diff-671d351f7832b66dcd494ce40ed352c6b49a22781ba08d4891670d3891bde599R51

but it's not clear to me how to apply this and how to not have it overwritten at each of the frequent homeassistant updates.

Is anyone more experienced and able to help?

Thanks

Hi,

You can find all the SIA codes on this page (sorry if it is in Spanish): https://doc.vanderbiltindustries.com/docs/intrusion/spc/spcpanel/v_3.11/installandconfig_olh/es/Content/Topics/SIA_Codes.htm

In my case I'm missing the "RL" code: Reader Locked

Ninja-ca avatar May 19 '25 17:05 Ninja-ca

Just going through the thread as I'm in the same boat as everyone else, has anyone tried any of the patches or the work around?

mr9lt avatar Jun 12 '25 14:06 mr9lt

its getting worse,

after migrate to 2025060

025-06-12 13:32:09.134 WARNING (ImportExecutor_0) [py.warnings] /usr/local/lib/python3.13/site-packages/pysiaalarm/data/data.py:7: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. import pkg_resources

aperezva avatar Jun 12 '25 14:06 aperezva

it seems nobody maintain this integration. Really a pity,

aperezva avatar Jun 12 '25 14:06 aperezva

so we have 2 solutions above:

ssh root@homeassistant -p 22222      
docker exec -it homeassistant sh
cd /usr/local/lib/python3.13/site-packages/pysiaalarm/data
 vi sia_codes.json

but then it's not clear to me what "missing sia codes" I should add (I believe there are 325 entries in that file already) apply the changes from here eavanvalkenburg/pysiaalarm@2e26496#diff-671d351f7832b66dcd494ce40ed352c6b49a22781ba08d4891670d3891bde599R51 but it's not clear to me how to apply this and how to not have it overwritten at each of the frequent homeassistant updates. Is anyone more experienced and able to help? Thanks

Hi,

You can find all the SIA codes on this page (sorry if it is in Spanish): https://doc.vanderbiltindustries.com/docs/intrusion/spc/spcpanel/v_3.11/installandconfig_olh/es/Content/Topics/SIA_Codes.htm

In my case I'm missing the "RL" code: Reader Locked

Quick query @Ninja-ca , from the link you provided, I assume you have managed to get this to work with a Vanderbilt SPC alarm? I have tried but failed to get correct settings in SPC panel and SIA integration, could you share your approach on settings?

fergalom avatar Jul 02 '25 13:07 fergalom

so we have 2 solutions above:

ssh root@homeassistant -p 22222      
docker exec -it homeassistant sh
cd /usr/local/lib/python3.13/site-packages/pysiaalarm/data
 vi sia_codes.json

but then it's not clear to me what "missing sia codes" I should add (I believe there are 325 entries in that file already) apply the changes from here eavanvalkenburg/pysiaalarm@2e26496#diff-671d351f7832b66dcd494ce40ed352c6b49a22781ba08d4891670d3891bde599R51 but it's not clear to me how to apply this and how to not have it overwritten at each of the frequent homeassistant updates. Is anyone more experienced and able to help? Thanks

Hi, You can find all the SIA codes on this page (sorry if it is in Spanish): https://doc.vanderbiltindustries.com/docs/intrusion/spc/spcpanel/v_3.11/installandconfig_olh/es/Content/Topics/SIA_Codes.htm In my case I'm missing the "RL" code: Reader Locked

Quick query @Ninja-ca , from the link you provided, I assume you have managed to get this to work with a Vanderbilt SPC alarm? I have tried but failed to get correct settings in SPC panel and SIA integration, could you share your approach on settings?

Hi @fergalom,

No, I have an Ajax System Alarm. SIA Protocol is standard.

To get the missing code you need to modify another file: base_server.py

Image

Use some way to access by SSH to your HA (in my case using "Advanced SSH & Web Terminal" - Disable "Protection Mode") Access to the HA docker: sudo docker exec -it homeassistant sh Go to the SIA Protocol Folder: cd /usr/local/lib/python3.13/site-packages/pysiaalarm/ Add missing SIA Codes: vi base_server.py Use "insert" button to start writting Search the line starting with: "if counter == COUNTER_CODE and event:" and modify the line bellows by: ""Code not found, replying with DUH to account: %s", event.account" Save & Exit: "Escape" button + :wq!

Image

With this, you will see the missing code in your System Logs

Ninja-ca avatar Jul 23 '25 10:07 Ninja-ca

https://github.com/home-assistant/core/issues/130207#issuecomment-3175016944

cddu33 avatar Aug 12 '25 16:08 cddu33