crawl4ai
crawl4ai copied to clipboard
feat/add Firecrawl backend support to crawler
I introduces Firecrawl as an optional backend for crawl4ai.
Updates
- Added FirecrawlBackend wrapper around Firecrawl’s SDK.
- Extended CLI with --backend option (default | firecrawl).
- Enabled output in multiple formats (json, markdown, markdown-fit).
- Added a standalone script (firecrawl_backend.py) showing how to use Firecrawl programmatically.
- Updated README with installation and usage instructions for Firecrawl.
Example usage : python -m crawl4ai.cli crawl https://docs.firecrawl.dev --backend firecrawl --output markdown