hrequests icon indicating copy to clipboard operation
hrequests copied to clipboard

Asyncio Error After Version 0.9.0 Update

Open zieddhf opened this issue 1 year ago • 6 comments

  • After updating the hrequests package to version 0.9.0, our application started crashing with the following error:

RuntimeError: asyncio.run() cannot be called from a running event loop

Full log error :

Downloading hrequests-cgo library from daijro/hrequests...
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.8/14.8 MB 9.8 MB/s
Traceback (most recent call last):
  File "/home/appuser/.local/bin/uvicorn", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/appuser/.local/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/appuser/.local/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/appuser/.local/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/appuser/.local/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/appuser/.local/lib/python3.12/site-packages/uvicorn/main.py", line 412, in main
    run(
  File "/home/appuser/.local/lib/python3.12/site-packages/uvicorn/main.py", line 579, in run
    server.run()
  File "/home/appuser/.local/lib/python3.12/site-packages/uvicorn/server.py", line 65, in run
    return asyncio.run(self.serve(sockets=sockets))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/appuser/.local/lib/python3.12/site-packages/uvicorn/server.py", line 69, in serve
    await self._serve(sockets)
  File "/home/appuser/.local/lib/python3.12/site-packages/uvicorn/server.py", line 76, in _serve
    config.load()
  File "/home/appuser/.local/lib/python3.12/site-packages/uvicorn/config.py", line 434, in load
    self.loaded_app = import_from_string(self.app)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/appuser/.local/lib/python3.12/site-packages/uvicorn/importer.py", line 19, in import_from_string
    module = importlib.import_module(module_str)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/app/main.py", line 12, in <module>
    from myapp.api import ar, router
  File "/app/myapp/api/__init__.py", line 2, in <module>
    from .dig import ar, router
  File "/app/myapp/api/dig.py", line 35, in <module>
    from ..excavators.company import Company, DomainName, company_by_domain
  File "/app/myapp/excavators/company.py", line 13, in <module>
    import hrequests
  File "/home/appuser/.local/lib/python3.12/site-packages/hrequests/__init__.py", line 33, in <module>
    from .session import Session, TLSSession, chrome, firefox
  File "/home/appuser/.local/lib/python3.12/site-packages/hrequests/session.py", line 5, in <module>
    from browserforge.headers import Browser as BFConstraints
  File "/home/appuser/.local/lib/python3.12/site-packages/browserforge/headers/__init__.py", line 3, in <module>
    DownloadIfNotExists()
  File "/home/appuser/.local/lib/python3.12/site-packages/browserforge/download.py", line 159, in DownloadIfNotExists
    Download()
  File "/home/appuser/.local/lib/python3.12/site-packages/browserforge/download.py", line 150, in Download
    asyncio.run(AsyncDownload(headers=headers, fingerprints=fingerprints))
  File "/usr/local/lib/python3.12/asyncio/runners.py", line 190, in run
    raise RuntimeError(
RuntimeError: asyncio.run() cannot be called from a running event loop
sys:1: RuntimeWarning: coroutine 'AsyncDownload' was never awaited

Downgrading to hrequests v0.8.1 resolves the issue, and the application runs without errors.

Environment:

  • Python version: 3.12

  • hrequests version: 0.9.0 (issue occurs with this version)

  • uvicorn: 0.32.0

  • OS: Debian (based on python:3.12-slim image)

  • Docker version: 27.3.1

zieddhf avatar Nov 14 '24 11:11 zieddhf

Hello, what version of Browserforge do you have installed?

python3 -m pip freeze | grep browserforge

daijro avatar Nov 14 '24 11:11 daijro

Hello, what version of Browserforge do you have installed?

python3 -m pip freeze | grep browserforge

Hello 👋 browserforge v1.1.2

zieddhf avatar Nov 14 '24 11:11 zieddhf

Any updates about this issue?

jpobletec avatar Nov 18 '24 19:11 jpobletec

Same issue here, browserforge version is also v1.1.2

anaselmhamdi avatar Jan 01 '25 20:01 anaselmhamdi

@jpobletec @anaselmhamdi Is it a fresh install or update like @zieddhf described? Also, please post logs if the issue is still relevant..

vitalis avatar Mar 03 '25 00:03 vitalis

Tried to install and use it within Google Colab and got this error when trying to import it. pip freeze shows hrequests==0.9.2

[/usr/local/lib/python3.11/dist-packages/hrequests/__init__.py](https://localhost:8080/#) in <module>
     31 from .reqs import *
     32 from .response import ProcessResponse, Response
---> 33 from .session import Session, TLSSession, chrome, firefox
     34 
     35 # attempt to import headless browsing dependencies

[/usr/local/lib/python3.11/dist-packages/hrequests/session.py](https://localhost:8080/#) in <module>
      3 from typing import Dict, Literal, Optional, Tuple, Union
      4 
----> 5 from browserforge.headers import Browser as BFConstraints
      6 from browserforge.headers import HeaderGenerator
      7 from ua_parser import user_agent_parser

[/usr/local/lib/python3.11/dist-packages/browserforge/headers/__init__.py](https://localhost:8080/#) in <module>
      1 from browserforge.download import DownloadIfNotExists
      2 
----> 3 DownloadIfNotExists()
      4 
      5 from .generator import Browser, HeaderGenerator

[/usr/local/lib/python3.11/dist-packages/browserforge/download.py](https://localhost:8080/#) in DownloadIfNotExists()
    157 def DownloadIfNotExists() -> None:
    158     if not IsDownloaded():
--> 159         Download()
    160 
    161 

[/usr/local/lib/python3.11/dist-packages/browserforge/download.py](https://localhost:8080/#) in Download(headers, fingerprints)
    148 def Download(headers=True, fingerprints=True) -> None:
    149     try:
--> 150         asyncio.run(AsyncDownload(headers=headers, fingerprints=fingerprints))
    151     except KeyboardInterrupt:
    152         print('Download interrupted.')

[/usr/lib/python3.11/asyncio/runners.py](https://localhost:8080/#) in run(main, debug)
    184     if events._get_running_loop() is not None:
    185         # fail fast with short traceback
--> 186         raise RuntimeError(
    187             "asyncio.run() cannot be called from a running event loop")
    188 

RuntimeError: asyncio.run() cannot be called from a running event loop

retkowski avatar Apr 16 '25 21:04 retkowski