botasaurus icon indicating copy to clipboard operation
botasaurus copied to clipboard

URLs in parallel using browser tabs and rotating proxies with early exists

Open RizwanSabir opened this issue 5 months ago • 1 comments

Scrapes URLs in parallel (3 at a time) using browser tabs and rotating proxies.

Searches for specific "success keywords" like "purchase", or "sale" on the page content.

If any one of the pages contains the success condition, it immediately:

Stops all other ongoing scrapes (early exit).

Extracts important data (like the heading and price).

Prints the first successful result, including how long the scraping took.

RizwanSabir avatar Jul 14 '25 16:07 RizwanSabir

how to do early exists

RizwanSabir avatar Jul 14 '25 16:07 RizwanSabir