crawl4ai
crawl4ai copied to clipboard
unclecode
🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN
Hi, Thanks for the great repository. I am new to this repository, I was curious to know if there is any support to change the language before I crawl a...
The code section of: https://crawl4ai.com/mkdocs/examples/summarization/ is broken, pls. fix it
screenshot=True, takes screenshot before wait_for finishes, so on webpages where data loads later it take screenshot of empty pages.
This PR adds proxy functionality to the AsyncWebCrawler and AsyncPlaywrightCrawlerStrategy classes. Linked Issue: #116 - Modified AsyncWebCrawler to accept a `proxy` parameter. - Updated AsyncPlaywrightCrawlerStrategy to handle proxy settings when...
I know libertarians will not be happy but ... In Europe, scrapping websites for the purpose of Text and Data Mining and LLM training is **legal** (this is the good...
Hi, when I try to run crawl4ai with microsoft edge on windows, I have this error below, ( same code works for ubuntu on chrome) Traceback (most recent call last):...
Hi, Since this is a recent repository, if someone wants to generate code that uses this library in either chatGPT or any other coding assistant it doesn't work. Would it...
I was going through prompt and i encounter spelling mistake in prompt so helping with it :)
Related to #105 Fix the 'NoneType' object has no attribute 'get' error in `AsyncWebCrawler`. * **crawl4ai/async_webcrawler.py** - Add a check in the `arun` method to ensure `html` is not `None`...
