crawl4ai

crawl4ai copied to clipboard

Before: YHere After: Here

tuhinmallick

Browser is not supported: Select playwright browser="firefox"

1

How can I select a browser that is supported by the target web site? Running this test in crawl4ai produces the incorrect login screen: ```py async def main(): crawler_strategy =...

dcolley

Hi, Is it possible to do smart crawl integrating LLM agent. For example, along with url, use can pass small instruction what actually user wants from webpage. based on instruction,...

Darshan2104

Traceback (most recent call last): File "/Users/youknowwho/Downloads/repos/ai/customllm/custom-llm/Automation_code/Web_crawling/crawl4ai.py", line 2, in import crawl4ai ^^^^^^^^^^^^^^^ File "/Users/youknowwho/Downloads/repos/ai/customllm/custom-llm/Automation_code/Web_crawling/crawl4ai.py", line 3, in from crawl4ai import AsyncWebCrawler as aicrawl4ai ImportError: cannot import name 'AsyncWebCrawler' from...

MSR2201

# Fix CSS selector in README example ## Description This pull request updates the CSS selector in the Advanced Usage example in the README.md file. The change fixes a non-functioning...

hitesh22rana

This code file: crawl4ai/web_crawler.py needs minor changes

1

1. Redundant kwargs in fetch_pages: The kwargs being passed in executor.map seem redundant, as they are being unpacked in the same format for every call. You can simplify this by...

vignesh1507

#133 resolved

vignesh1507

Cache Optionality

2

Allow the user to choose which folder to cache contents

landmann

Bypassing automated crawler detection by Firewalls

2

Is there a solution for websites behind WAFs like PerimeterX, Cloudflare, Akmai etc.?

dnmahendra

Hi, there , Is there a Docker deployment version of Crawl4ai and a usage method for the REST API?

socekin