crawl4ai
crawl4ai copied to clipboard

Published 20 hours ago •

Reame
Issues

Detect different failures cases

Open si2w opened this issue 4 months ago • 0 comments

It would be useful as well to detect pages with:

Captcha
Bot wall
Please update your browser
Parked domain
...

Those errors are already handled if I'm not wrong :

HTTP errors (4xx, 5xx)
TLS issue

Oct 26 '24 06:10 si2w