WARNING: Retrying crawlera request for authentication issue

[redjonmata]

I've trying to use scrapy-crawlera as a proxy for scraping some data. I've added these rows in settings.py:

DOWNLOADER_MIDDLEWARES = { 'scrapy_crawlera.CrawleraMiddleware': 610, } CRAWLERA_ENABLED = True CRAWLERA_APIKEY = 'MY_CRAWLERA_API_KEY'

In zyte.com i have created a Smart proxy manager account and my api key is from my account settings.

I get this warning WARNING: Retrying crawlera request for authentication issue and the websites doesn't get scraped.

Is there something I'm doing wrong?

[ilias-ant]

I am also getting this WARNING message multiple times for multiple spiders (although i think it does not impact the actual scraping functionality).

I have also opened a ticket in the support.zyte.com, displaying the issue as most probably originates from the core Smart Proxy Manager API and not from this library, which is merely a client (ofc this is just an educated guess). Will let you know if something interesting arises from there. :)

[bishalnepali]

I encountered this today. Any solution you guys applied?

[Gallaecio]

my api key is from my account settings

It sounds like you could be using your Zyte dashboard / Scrapy Cloud API key, which is not the same as your Smart Proxy Manager API key (you can in fact have multiple Smart Proxy Manager API keys).

You can find your Smart Proxy Manager API key after you select a specific Smart Proxy Manager account from the dashboard.

If you are sure you are using the right API key, you should open a support ticket to get it resolved quickly.

[bishalnepali]

I am sure that I am using Smart Proxy Manager API key. Will open the ticket Thanks!

[Gallaecio]

Closing, as it seems opening a support ticket would be the right solution if it ever happens to anyone.