crawl4ai

crawl4ai copied to clipboard

Langchain - Document loader basic functionality i.e `lazy load` implemented Issue #77

2

aravindkarnam

Crawling error

7

hey and thanks for this nice package! am having the following issue: some websites are randomly not scraped, while others get scraped correctly. On each run of the code which...

BlackChila

429 rate limit of 1 request per minute

2

I switched to the latest docker image and started to see the 429 error code, when crawl4ai is saying rate limit of request is 1 per minute. However, such rate...

monkeybiz55

I just wonder in doc where is AsyncWebCrawler

4

so the async method is cancel?

NicoNicoNico123

Has the CosineStrategy been deleted?

1

Has the CosineStrategy been deleted?

1933211129

No way to set crawl depth for crawler

1

Currently crawler only crawls links of depth level 1. That means if you give homepage link (homepage.com) it will only crawl direct links from that homepage and it will not...

matijaparavac

Extracting data from an iframe?

1

What is the best method for extracting data from an iframe using Crawl4ai? Here is an example of the iframe I am trying to capture: `$1757 / monthAvailable: Now231 Crestview...

ehubb20

Crawl4ai Hard dependencites

1

Hi guys, I started embedding your lib to my project, when i added it to requirements i started getting non stop errors on dependencies: The conflict is caused by: The...

TomerLankri

Docker Issue: ModuleNotFoundError: No module named 'numpy'

3

I am running the project as a Docker container and I get "ModuleNotFoundError: No module named 'numpy'". I followed these instructions to bring it up: https://crawl4ai.com/mkdocs/installation/ ``` # For other...

mjtechguy

Dears, while using your library, I needed to access the raw content of requests being sent when a page is crawled (for instance, requests made to other APIs or files,...

ifaddict1