deep-searcher icon indicating copy to clipboard operation
deep-searcher copied to clipboard

Can not use deepsearcher load to load uri

Open JPHu3312 opened this issue 11 months ago • 3 comments

ERROR INF

(.venv) [root@aigc deep-searcher]# deepsearcher load "https://redis.io/docs/latest/operate/oss_and_stack/install/install-redis/install-redis-on-linux/"
Embedding chunks: 0it [00:00, ?it/s]
Exception ignored in: <function BaseSubprocessTransport.__del__ at 0x7f0b2af4b880>
Traceback (most recent call last):
  File "/usr/local/lib/python3.13/asyncio/base_subprocess.py", line 130, in __del__
  File "/usr/local/lib/python3.13/asyncio/base_subprocess.py", line 107, in close
  File "/usr/local/lib/python3.13/asyncio/unix_events.py", line 802, in close
  File "/usr/local/lib/python3.13/asyncio/unix_events.py", line 788, in write_eof
  File "/usr/local/lib/python3.13/asyncio/base_events.py", line 833, in call_soon
  File "/usr/local/lib/python3.13/asyncio/base_events.py", line 556, in _check_closed
RuntimeError: Event loop is closed

config.yaml

web_crawler:
#    provider: "FireCrawlCrawler"
#    config: {}

     provider: "Crawl4AICrawler"
     config: # Uncomment to custom browser configuration for Crawl4AI
       browser_config:
         headless: true
#         proxy: "http://127.0.0.1:7890"
#         chrome_channel: "chrome"
         verbose: true
#         viewport_width: 800
#         viewport_height: 600

#    provider: "JinaCrawler"
#    config: {}

What I can do for this error?

JPHu3312 avatar Mar 20 '25 04:03 JPHu3312

@jinhonglin-ryan could you please take a look

zc277584121 avatar Mar 20 '25 12:03 zc277584121

Hi @JPHu3312 Sorry for the frustration. I have tested crawling this website using Crawl4AI and have the exact same setup as you did, but I did not encounter this error.

Image

I am wondering have you run crawl4ai-setup in your running environment before? If not, please run it, as it will set up crawl4ai ready for you. If this issue persists after you run the command. Feel free to share more details to us and we are willing to help! Thanks!

jinhonglin-ryan avatar Mar 31 '25 05:03 jinhonglin-ryan