crawl4ai icon indicating copy to clipboard operation
crawl4ai copied to clipboard

Adding a parameter to disable SSL verification during crawling

Open ifaddict1 opened this issue 1 year ago • 0 comments

Dears, while building a tool with your wonderful library, I realized there was no support for disabling SSL verification for websites that either use HTTP only or use HTTPS with an invalid certificate. So I added this option in the "AsyncPlaywrightCrawlerStrategy" class. I think this change could benefit a lot of people and prevent breaking certain scripts. I also added a test. Let me know if I should modify anything

ifaddict1 avatar Sep 29 '24 17:09 ifaddict1