crawl4ai
crawl4ai copied to clipboard
unclecode
🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN
### crawl4ai version 0.7.7 ### Expected Behavior When using the embedding strategy in AdaptiveCrawler, the system should: 1. Take the user's query (e.g., "who is on the board of directors")...
### crawl4ai version 0.7.6 ### Expected Behavior `AsyncUrlSeeder._resolve_head(url)` should return `None` when `url` doesn't resolve ### Current Behavior `AsyncUrlSeeder._resolve_head(url)` doesn't return `None` when `url` doesn't resolve. ```python async def _resolve_head(self,...
### crawl4ai version latest ### Expected Behavior light_mode=False navigator is dark mode ### Current Behavior Hello i'm running wth config config = BrowserConfig( headless=False, light_mode=False, enable_stealth=True, headers={ "Accept-Language": "fr-FR,fr;q=0.9", "Accept":...
--- ## **Summary** Fixes #1631 This PR fixes the issue where `light_mode=False` did not enable dark mode correctly and instead acted as a performance configuration flag. The browser initially launched...
# Crawl4AI Enhanced Features - Open Source Contribution ## Contribution Overview This contribution adds production-grade security, performance, and operational features to Crawl4AI, enabling it to handle enterprise workloads of 500+...
### crawl4ai version 0.7.6 ### Expected Behavior # crawler_pool.py (new file) import asyncio, json, hashlib, time, psutil from contextlib import suppress from typing import Dict from crawl4ai import AsyncWebCrawler, BrowserConfig...
### crawl4ai version 0.7.6 ### Expected Behavior Get dynamic web content ### Current Behavior Hello, here's the situation: I am using the following method to scrape web pages, but dynamic...
### crawl4ai version Crawl4AI 0.5.0.post8 ### Expected Behavior i was using your pip package perfectly fine but after the Crawl4AI 0.5.0.post8 update after the every 5 ferquest its giving a...
### crawl4ai version 0.7.6 ### Expected Behavior When running crawl4ai from the docker image with a security config of `security: enabled: true jwt_enabled: true ` I would expect it is...
### crawl4ai version 0.6.3 ### Expected Behavior I made the following code settings to implement a POST request for crawl4ai. However, the backend server doesn't receive any values submitted via...
