Optimize scraping with threads

[Crinibus]

Update so that scraping at different domains can happen at the same time, but scraping a each domain is sequentially

E.g. if there is four products to be scraped from two different domains, then one product from each domain is scraped at the same time, then after the request_delay the next product from each domain is being scraped at the same time.