crawl4ai
crawl4ai copied to clipboard
unclecode
🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN
### crawl4ai version 0.6.3 ### Expected Behavior Relative links on a page I was redirected to should use the base url from the page that the browser was redirected to,...
### crawl4ai version 0.7.6 ### Expected Behavior Same as other people I am not able to get llm extraction to run. I have setup via docker compose and want to...
Basic Health Check Endpoint to verify if a URL is accessible before running a full crawl operation
## Summary Please include a summary of the change and/or which issues are fixed. eg: `Fixes #123` (Tag GitHub issue numbers in this format, so it automatically links the issues...
### crawl4ai version 0.7.0 ### Expected Behavior When called upon, MCP SSE server responds properly ### Current Behavior MCP server is added to Claude Code and shows as "connected", however,...
### crawl4ai version 0.7.6 ### Expected Behavior Adaptive crawler. `strategy="embedding",` `embedding_model="openai/text-embedding-3-small", embedding_llm_config=LLMConfig( provider="openai/gpt-4o-mini", // openai/text-embedding-3-small fails here too api_token=OPENAI_API_KEY, ),` Query: 'capabilities, services, certifications, description of work, products' Query Expansion:...
### crawl4ai version 0.7.6 ### Expected Behavior using the LLMExtractionStrategy, model choice gemini ### Current Behavior When using the LLMExtractionStrategy, if the maxtokens are set too small, the context window...
### crawl4ai version 0.7.4 ### Expected Behavior Expected to see html with classes and ids, otherwise i cant use it for further analyzing ### Current Behavior No classes/ids ### Is...
### crawl4ai version 0.6.3 ### Expected Behavior The asynchronous generator works fine ### Current Behavior Hello, when I started a web service and ran the deep search function multiple times,...
## Summary Please include a summary of the change and/or which issues are fixed. eg: `Fixes #123` (Tag GitHub issue numbers in this format, so it automatically links the issues...
### crawl4ai version 0.6.3 ### Expected Behavior the url is not able to scrape i am getting below error Unexpected error in _crawl_web at line 744 in _crawl_web (.venv/lib/python3.11/site-packages/crawl4ai/async_crawler_strategy.py ###...
