news-please icon indicating copy to clipboard operation
news-please copied to clipboard

Proxy Server configuration (HttpProxyMiddleware)

Open bkrishnap opened this issue 3 years ago • 4 comments

Is there a way we can configure the proxy server with news-please? I would like to add a proxy endpoint to hide the main IP of the server that is installed with news-please.

I tried to figure out where I can fit it with the HttpProxyMiddleware with the help of below documentation and I am not successful with the configs.

https://doc.scrapy.org/en/1.1/topics/downloader-middleware.html#module-scrapy.downloadermiddlewares.httpproxy

bkrishnap avatar Aug 07 '22 13:08 bkrishnap

did you check whether the requests library supports this?

fhamborg avatar Jan 14 '23 14:01 fhamborg

requests does support. Something like this? But where should be entrypoint for this ..

`import requests

proxies = { "http": "http://10.10.1.10:3128", "https": "http://10.10.1.10:1080", }

requests.get("http://example.org", proxies=proxies)`

bkrishnap avatar Jan 17 '23 22:01 bkrishnap

I'm curious about this as well.

mcnamara-charles avatar Aug 17 '23 17:08 mcnamara-charles