scrapy-zyte-smartproxy

scrapy-zyte-smartproxy copied to clipboard

Support easy session reuse

2

_May_ solve #27. It definitely allows removing some boilerplate.

Gallaecio

WARNING: Retrying crawlera request for authentication issue

1

I've trying to use scrapy-crawlera as a proxy for scraping some data. I've added these rows in settings.py: `DOWNLOADER_MIDDLEWARES = { 'scrapy_crawlera.CrawleraMiddleware': 610, } CRAWLERA_ENABLED = True CRAWLERA_APIKEY = 'MY_CRAWLERA_API_KEY'`...

redjonmata

Blacklist domains

1

I was setuping autoextract in scrapy cloud on a project with crawlera addon. Autoextract queries were routed through crawlera. Idea is to blacklist autoextract domain by default. It may have...

whalebot-helmsman

breaks `response.text` for other middlewares

1

calling `response.text` in any download middleware loaded after scrapy_crawlera middleware results in `*** AttributeError: Response content isn't text`

getorca

Support more secure ways to declare the APIKEY

1

### BACKGROUND: As of version `1.6.0`, there are two (2) ways of adding the API KEYS: 1. via the `settings.py`: ``` CRAWLERA_APIKEY = 'apikey' ``` 2. via spider attribute: ```...

BurnzZ

Native support for X-Crawlera-Session

11

It'd be great if the plugin can be configured that it'll use/re-use the sessions mechanism. Because managing it in spiders like that: ``` if 'X-Crawlera-Session' in response.headers and response.headers['X-Crawlera-Session'] !=...

dchaplinsky

scrapy_zyte_smartproxy in version 2.2.0 issue with request

1

When deploying to our server with scrapy_zyte_smartproxy version 2.2.0 encountered an issue when trying to do a request and it keeps on retrying until it gave up. What I did...

donahcandia0121

After v2.3.0 I'm getting timeouts to URLs

2

Not sure how to help diagnose this, but we started getting Timeout Errors after we started using `2.3.0`. `2.3.1` doesn't solve the issue either, but downgrading to `v2.2.0` works. Please...

sebastian-correa