external proxy support

[vtbalaji]

thanks for the worker component, it fits my needs and loving it.

I link to know how to use "worker component" with external proxy.

Any tips or idea will be appreciated.

thanks in advance.

[ziflex]

Could you elaborate on that? Proxy for CDP or InMemory driver? How do you run it?

If you are using CDP driver and are running the worker within an official Docker image, then you need to launch the image with this flag -e CHROME_OPTS='--proxy-server=my-proxy.com', where my-proxy.com is your target proxy server address.

If you are using in memory driver, then I will need to add this option to the worker.

[vtbalaji]

hi Ziflex, thx for quick response. we are using CDP driver and running official Docker image without any change so far. i tried running worker component as recommended by you with below command docker run -p 80:8080 -e CHROME_OPTS='http://key:@proxy.crawlera.com:8011/' montferret/worker

it does not seem to be working or even reaching proxy. Am i missing anything here.

regards Balaji

[ziflex]

hi Ziflex, thx for quick response. we are using CDP driver and running official Docker image without any change so far. i tried running worker component as recommended by you with below command docker run -p 80:8080 -e CHROME_OPTS='http://key:@proxy.crawlera.com:8011/' montferret/worker

it does not seem to be working or even reaching proxy. Am i missing anything here.

regards Balaji

Hey, I think you missed something, it should be like this:

docker run -p 80:8080 -e CHROME_OPTS='--proxy-server=http://key:@proxy.crawlera.com:8011/' montferret/worker

[vtbalaji]

Hi Ziflex, thx for your support, i am running following command docker run -p 80:8080 -e CHROME_OPTS='--proxy-server=http://xyzkey:@proxy.crawlera.com:8011/' montferret/worker

Getting following error

run program: net::ERR_NO_SUPPORTED_PROXIES: DOCUMENT(@url,{driver:'cdp'})

if i remove proxy and provide direct access things are working fine.

Thanks and Regards Balaji