Remote Driver with Selenium Grid in Docker

[IOANNB4R]

I'm trying to use Selenium Grid in Docker with selenium-wire but unfortunately, it doesn't work. It works with the general Selenium library though:

from selenium import webdriver

driver = webdriver.Remote('http://127.0.0.1:4444', options=webdriver.ChromeOptions())

driver.get('https://python.org')
driver.save_screenshot('screenshot.png')

driver.quit()

But when I use selenium-wire, it just gets stuck. I use the following code:

from seleniumwire import webdriver

driver = webdriver.Remote('http://127.0.0.1:4444', options=webdriver.ChromeOptions())

driver.get('https://python.org')
driver.save_screenshot('screenshot.png')

driver.quit()

Also, I noticed: when I use Selenium, the Grid Panel looks like this:

But when I use selenium-wire, it looks like that:

The full JSON string it shows:

{
  "acceptInsecureCerts": true,
  "browserName": "firefox",
  "goog:chromeOptions": {
    "extensions": [],
    "args": []
  },
  "moz:debuggerAddress": true,
  "pageLoadStrategy": "normal",
  "proxy": {
    "proxyType": "manual",
    "httpProxy": "[::ffff:127.0.0.1]:50540",
    "sslProxy": "[::ffff:127.0.0.1]:50540"
  }
}

[felipeacunago]

Hi!,

you need to set the proxy server to point the VM/docker that runs the code. For example your code should be:

from selenium import webdriver

chrome_options = webdriver.ChromeOptions()
# you are going to tell selenium the IP where the code will run
# replace scrap with the IP, I use scrap because my docker-compose file uses that name
chrome_options.add_argument('--proxy-server=scrap:8087')
chrome_options.add_argument('--ignore-certificate-errors')

# replace chrome with the IP of the selenium docker, I use chrome because my docker-compose file uses that name
driver = webdriver.Remote(
    command_executor='http://chrome:4444/wd/hub',
    desired_capabilities=chrome_options.to_capabilities(),
    seleniumwire_options={
        'auto_config': False,
        'addr': '0.0.0.0',
        'port': 8087
    })

driver.get('https://python.org')
driver.save_screenshot('screenshot.png')

driver.quit()

And then you should expose that port, I usually run this in a docker-compose environment so I do it like this:

version: '3'

services:
  scrap: #image build using your code
    build:
      context: .
      dockerfile: ./scrap/Dockerfile
    ports:
      - "8087:8087" # I expose the same port I used as proxy in the previous step
      
  chrome: #selenium image
    image: selenium/standalone-chrome:latest
    ports:
      - "4444:4444"
    shm_size: '2gb'

Check this https://github.com/wkeeling/selenium-wire/issues/627 for more information. Hope it helps!

[User-Clb]

Hi!,

you need to set the proxy server to point the VM/docker that runs the code. For example your code should be:

from selenium import webdriver

chrome_options = webdriver.ChromeOptions()
# you are going to tell selenium the IP where the code will run
# replace scrap with the IP, I use scrap because my docker-compose file uses that name
chrome_options.add_argument('--proxy-server=scrap:8087')
chrome_options.add_argument('--ignore-certificate-errors')

# replace chrome with the IP of the selenium docker, I use chrome because my docker-compose file uses that name
driver = webdriver.Remote(
    command_executor='http://chrome:4444/wd/hub',
    desired_capabilities=chrome_options.to_capabilities(),
    seleniumwire_options={
        'auto_config': False,
        'addr': '0.0.0.0',
        'port': 8087
    })

driver.get('https://python.org')
driver.save_screenshot('screenshot.png')

driver.quit()

And then you should expose that port, I usually run this in a docker-compose environment so I do it like this:

version: '3'

services:
  scrap: #image build using your code
    build:
      context: .
      dockerfile: ./scrap/Dockerfile
    ports:
      - "8087:8087" # I expose the same port I used as proxy in the previous step
      
  chrome: #selenium image
    image: selenium/standalone-chrome:latest
    ports:
      - "4444:4444"
    shm_size: '2gb'

Check this #627 for more information. Hope it helps!

Hi,I don't quite understand "webdriver. Remote" to run in docker? If I don't run a browser in docker environment that can "Remote" selenium Grid, I don't know if that makes sense