node-scraper icon indicating copy to clipboard operation
node-scraper copied to clipboard

trying to fetch ñ.ñ

Open jcarlos7121 opened this issue 12 years ago • 0 comments

i used an array of urls to scrape, the number of urls i need to fetch are around 7500 urls ñ.ñ, i begin scraping and everything goes fine until the first 120 urls fetched, it starts going slooooooow... more and more, until it crashes and cannot fetch another one anymore, does the library supports heavy scraping i need to scrape tons of things :(

jcarlos7121 avatar Dec 09 '13 23:12 jcarlos7121