[🐛 Bug]: Grid 4.4.0 - Could not start a new session

[PatoSoft]

What happened?

I am getting this error:

Selenium::WebDriver::Error::SessionNotCreatedError: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: unknown error: Chrome failed to start: crashed.

• My host machine is a M1 Macbook pro running macOS Monterey
• My docker image uses ruby:3.1.2-slim-buster
• It is a Sinatra (Ruby) project

The error appear when creating the driver:

options = Selenium::WebDriver::Options.chrome
options.add_argument('--ignore-certificate-errors')
options.add_argument('--headless')
options.add_argument('--disable-gpu')
options.add_argument('--remote-debugging-port=9222')
options.add_argument('--disable-extensions')
options.add_argument('--disable-extensions')
options.add_argument('--disable-dev-shm-usage')

Selenium::WebDriver.for(
  :remote,
  {
    url: ENV.fetch('SELENIUM_DRIVER_URI'), # http://selenium_hub:4444/wd/hub/ in docker-compose
    capabilities: options
  }
)

I can see the hub at http://localhost:4444/, the session is there too but it fails just after get picked

Command used to start Selenium Grid with Docker

version: '3.9'
services:
  app:
    build:
      context: .
      dockerfile: Dockerfile.dev
    command: bundle exec puma -C config/puma.rb
    stdin_open: true
    tty: true
    depends_on:
      - selenium_hub
      - selenium_node_1
    volumes:
      - .:/app
    ports:
      - '3000:3000'
    environment:
      PUMA_BINDING: tcp://0.0.0.0:3000
      SELENIUM_DRIVER_URI: http://selenium_hub:4444/wd/hub/
  selenium_hub:
    image: selenium/hub:4.4.0
    ports:
      - 4442:4442
      - 4443:4443
      - 4444:4444
    environment:
      GRID_BROWSER_TIMEOUT: 90
      GRID_NEW_SESSION_WAIT_TIMEOUT: 15000
    volumes:
      - /dev/shm:/dev/shm
  selenium_node_1:
    image: selenium/node-chrome:4.4.0
    shm_size: 2gb
    depends_on:
      - selenium_hub
    environment:
      - SE_EVENT_BUS_HOST=selenium_hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_OVERRIDE_MAX_SESSIONS=true
      - SE_NODE_MAX_SESSIONS=10
    volumes:
      - /dev/shm:/dev/shm
    ports:
      - "6900:5900"

Relevant log output

Selenium::WebDriver::Error::SessionNotCreatedError: Could not start a new session. Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: unknown error: Chrome failed to start: crashed.
  (chrome not reachable)
  (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Build info: version: '4.4.0', revision: 'e5c75ed026a'
System info: host: '6e1f9cbe76df', ip: '192.168.32.3', os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.104-linuxkit', java.version: '11.0.16'
Driver info: driver.version: unknown
Build info: version: '4.4.0', revision: 'e5c75ed026a'
System info: host: '6e1f9cbe76df', ip: '192.168.32.3', os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.104-linuxkit', java.version: '11.0.16'
Driver info: driver.version: unknown
Build info: version: '4.4.0', revision: 'e5c75ed026a'
System info: host: '2da59887ca1b', ip: '192.168.32.2', os.name: 'Linux', os.arch: 'amd64', os.version: '5.10.104-linuxkit', java.version: '11.0.16'
Driver info: driver.version: unknown
from [remote server] org.openqa.selenium.grid.node.remote.RemoteNode(RemoteNode.java):150:in `newSession'

Operating System

M1 Macbook pro running macOS Monterey

Docker Selenium version (tag)

4.4.0

[PatoSoft]

I am running the same test on Staging which host is Ubuntu 18.04 LTS instead of macOS Monterey(arm) and I am getting this error instead:

Failure/Error:
  Selenium::WebDriver.for(
    :remote,
    {
      url: ENV.fetch('SELENIUM_DRIVER_URI'),
      capabilities:
    }
  )

NoMethodError:
  undefined method `closed?' for nil:NilClass

        if @socket.closed?
                  ^^^^^^^^
./vendor/bundle/ruby/3.1.0/gems/webmock-3.18.1/lib/webmock/http_lib_adapters/net_http.rb:92:in `block in request'
./vendor/bundle/ruby/3.1.0/gems/webmock-3.18.1/lib/webmock/http_lib_adapters/net_http.rb:97:in `request'
./vendor/bundle/ruby/3.1.0/gems/selenium-webdriver-4.4.0/lib/selenium/webdriver/remote/http/default.rb:118:in `response_for'
./vendor/bundle/ruby/3.1.0/gems/selenium-webdriver-4.4.0/lib/selenium/webdriver/remote/http/default.rb:77:in `request'
./vendor/bundle/ruby/3.1.0/gems/selenium-webdriver-4.4.0/lib/selenium/webdriver/remote/http/common.rb:59:in `call'
./vendor/bundle/ruby/3.1.0/gems/selenium-webdriver-4.4.0/lib/selenium/webdriver/remote/bridge.rb:625:in `execute'
./vendor/bundle/ruby/3.1.0/gems/selenium-webdriver-4.4.0/lib/selenium/webdriver/remote/bridge.rb:52:in `create_session'
./vendor/bundle/ruby/3.1.0/gems/selenium-webdriver-4.4.0/lib/selenium/webdriver/common/driver.rb:323:in `block in create_bridge'
./vendor/bundle/ruby/3.1.0/gems/selenium-webdriver-4.4.0/lib/selenium/webdriver/common/driver.rb:321:in `create_bridge'
./vendor/bundle/ruby/3.1.0/gems/selenium-webdriver-4.4.0/lib/selenium/webdriver/common/driver.rb:74:in `initialize'
./vendor/bundle/ruby/3.1.0/gems/selenium-webdriver-4.4.0/lib/selenium/webdriver/remote/driver.rb:43:in `initialize'
./vendor/bundle/ruby/3.1.0/gems/selenium-webdriver-4.4.0/lib/selenium/webdriver/common/driver.rb:57:in `new'
./vendor/bundle/ruby/3.1.0/gems/selenium-webdriver-4.4.0/lib/selenium/webdriver/common/driver.rb:57:in `for'
./vendor/bundle/ruby/3.1.0/gems/selenium-webdriver-4.4.0/lib/selenium/webdriver.rb:89:in `for'
./lib/smart_scraper.rb:47:in `bsd'
./lib/smart_scraper.rb:36:in `driver'
./lib/smart_scraper.rb:26:in `ensure in execute!'
./lib/smart_scraper.rb:26:in `execute!'
./spec/lib/smart_scraper_spec.rb:12:in `block (5 levels) in <top (required)>'
./vendor/bundle/ruby/3.1.0/gems/vcr-6.1.0/lib/vcr/util/variable_args_block_caller.rb:9:in `call_block'
./vendor/bundle/ruby/3.1.0/gems/vcr-6.1.0/lib/vcr.rb:194:in `use_cassette'
./spec/lib/smart_scraper_spec.rb:11:in `block (4 levels) in <top (required)>'
./vendor/bundle/ruby/3.1.0/gems/webmock-3.18.1/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'
------------------
--- Caused by: ---
NoMethodError:
  undefined method `closed?' for nil:NilClass
  
        if @socket.closed?
                  ^^^^^^^^
  ./vendor/bundle/ruby/3.1.0/gems/webmock-3.18.1/lib/webmock/http_lib_adapters/net_http.rb:92:in `block in request'

I moved webmock gem to the top of the Gemfile just in case it wasn't loaded at that point but I get the same error.

This is my Gemfile now:

source 'https://rubygems.org'

gem 'puma'
gem 'sinatra'

gem 'webmock', groups: [:development, :test]

gem 'addressable'
gem 'selenium-webdriver', '4.4.0'

gem 'decontaminator'
gem 'mini_racer'
gem 'oga'

gem 'bugsnag'
gem 'dotenv', groups: [:development, :test]
gem 'influxdb'
gem 'utf8-cleaner'

group :development, :test do
  gem 'house_style'
  gem 'pry'
  gem 'rack-test'
  gem 'rspec'
  gem 'rspec_junit_formatter'

  gem 'simplecov'
  gem 'timecop'
  gem 'vcr'
end

[jamesmortensen]

Hi @PatoSoft if you're running on a Mac M1 (aarch64) then you'll need to use the experimental Seleniarm container images. See https://github.com/seleniumhq-community/docker-seleniarm for more details.

I see in your docker-compose.yml you're using selenium/node-chrome and hub, but I think you need seleniarm/hub and seleniarm/node-chromium instead. Does this solve your issue?

[PatoSoft]

Hi @jamesmortensen I tried seleniarm and it worked fine locally (I had a timeout error but this one was solved) the problem is I am using the Mac M1 (aarch64) for development but an Ubuntu machine for circleCI, staging and production environments.

It is there any way I can make it work for arm and non-arm hosts at the same time?

[jamesmortensen]

@PatoSoft the seleniarm containers are multi arch, so they should run on both aarch64 and x86_64. But if not, you can always detect what environment you are on with arch and various other shell commands and use the appropriate docker compose yaml for that platform.

You also mentioned CircleCI. They have arm runners. You can check out the circleci config file in the seleniarm repo. We use arm-medium runners to test the arm containers so you could theoretically use that option for CI as well.

Hope this helps!

[PatoSoft]

Thank you for the information. I will try that!

[jamesmortensen]

I'm going to close this issue. If the above information hasn't solved the problem, please do let us know!