crawl4ai
crawl4ai copied to clipboard
Adding a parameter to disable SSL verification during crawling
Dears, while building a tool with your wonderful library, I realized there was no support for disabling SSL verification for websites that either use HTTP only or use HTTPS with an invalid certificate. So I added this option in the "AsyncPlaywrightCrawlerStrategy" class. I think this change could benefit a lot of people and prevent breaking certain scripts. I also added a test. Let me know if I should modify anything