crawlera-tools

crawlera-tools copied to clipboard

I am crawling the job-posting of linkedin with the crawlera enabled. But the most of the time i am getting the respose code 503 and 500. i have even added...

Waqar-ali141

Possibly unclean threads termination

3

Alain Quenneville seen the following exception (running python 2.7.3 on Linux Ubuntu 12.04.3 LTS): ``` Exception in thread Thread-1 (most likely raised during interpreter shutdown): Traceback (most recent call last):...

qrilka

Specifying extra headers from commind line

25

- Added a function which generates random ua headers for each request - Hardcoded X-Crawlera-UA header for each request

oltarasenko