search-engine-parser icon indicating copy to clipboard operation
search-engine-parser copied to clipboard

Published 20 hours ago • verified publisher icon bisohns

Prevent suspicious flagging

Open deven96 opened this issue 5 years ago • 1 comments

Apply techniques to prevent search engine scraper from being flagged as suspicious activity. Randomly cycle through browser headers for scraping

deven96 avatar Feb 03 '19 00:02 deven96

Maybe add ability to provide a user agent and proxy too! I think it would be really cool.

AvinashReddy3108 avatar Oct 02 '20 07:10 AvinashReddy3108