crawl4ai
crawl4ai copied to clipboard
unclecode
🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN
### crawl4ai version 0.6.3 ### Expected Behavior Fixed the issue of arun_many() Method ### Current Behavior The same configuration can get the correct results when using arun(), and the result...
### crawl4ai version 0.7.2 ### Expected Behavior It opens the browser to loading state and always getting an error. ### Current Behavior I'm having this kind of issue and can't...
### crawl4ai version 0.7.7 ### Expected Behavior The crawl4ai server can decode ContentRelevanceFilter and execute deep crawls accordingly. ### Current Behavior The REST API returns status code 500, and I...
### crawl4ai version current master ### Expected Behavior https://github.com/unclecode/crawl4ai/blob/02f3127deda707b948e1970699fceae214677b86/crawl4ai/utils.py#L1645C5-L1645C17 it can not be changed by outside function ### Current Behavior it can not be changed by outside function ### Is...
## Summary This PR: - Extends LLMConfig with backoff delay/attempt/factor fields and thread them through LLMExtractionStrategy, LLMContentFilter, table extraction, and Docker API handlers - Exposes the backoff parameter knobs on...
> Here's a code snippet shared by one of our community members on discord. I could verify that it works, so you could try this out. Closing this issue. >...
https://github.com/unclecode/crawl4ai/blob/0024c82cdcbd7c2d9e9e93ec40c8fec2563eff8f/crawl4ai/async_crawler_strategy.py#L491C46-L491C76 Is there any implementation for the `_generate_screenshot_from_html()` method under the `AsyncPlaywrightCrawlerStrategy` class in **[crawl4ai/async_crawler_strategy.py](https://github.com/unclecode/crawl4ai/blob/0024c82cdcbd7c2d9e9e93ec40c8fec2563eff8f/crawl4ai/async_crawler_strategy.py)**? I ran into an error when I'd set up this `CrawlerRunConfig` - ```py run_config =...
## Summary Fixed a critical bug in AsyncUrlSeeder where `_resolve_head()` was incorrectly returning redirect targets without verifying they were alive. This could cause dead URLs to be treated as valid...
#1563 Fix memory leaks and race conditions in CDP managed browser crawling Fix memory leaks and race conditions when using arun_many() with managed CDP browsers. Each crawl now gets proper...
