System tests does not work with Chrome/Chromium out of the box

[runephilosof-abtion]

The guides https://guides.rubyonrails.org/testing.html does not mention what you need to install in order to run the system specs.

For :headless_firefox, it is enough to install firefox-esr. But testing with :headless_chrome fails, even though the browser is installed.

The guides should list prerequisites. It would also be awesome, if the CI tested this, either as is done below or by adding to https://github.com/rails/rails/blob/main/.github/workflows/rails-new-docker.yml

Steps to reproduce

Dockerfile

FROM ruby:3.0.6

RUN curl -so nodejs.tar.xz https://nodejs.org/dist/v18.14.1/node-v18.14.1-linux-x64.tar.xz \
  && tar xJf nodejs.tar.xz -C /usr/local --strip-components=1 --no-same-owner \
  && node --version \
  && npm --version

RUN npm install -g yarn \
  && yarn --version

docker build -t rails-test-system .
docker run --rm -it rails-test-system bash
apt update
apt install -y chromium firefox-esr
gem install rails
rails new test-system
cd test-system
bin/rails g scaffold article
bin/rails db:migrate
sed -i 's/:chrome/:headless_firefox/' test/application_system_test_case.rb
bin/rails test:system
sed -i 's/:headless_firefox/:headless_chrome/' test/application_system_test_case.rb
bin/rails test:system

wget -q https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
apt-get install -y ./google-chrome-stable_current_amd64.deb
bin/rails test:system

Expected behavior

The system tests should be successful, like when using :headless_firefox.

Actual behavior

Using chromium

Error:
ArticlesTest#test_should_destroy_Article:
Selenium::WebDriver::Error::SessionNotCreatedError: session not created: Chrome failed to start: exited normally.
  (session not created: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
    #0 0x5c7383be6fb3 <unknown>

Using google-chrome

Error:
ArticlesTest#test_should_update_Article:
Selenium::WebDriver::Error::SessionNotCreatedError: session not created: Chrome failed to start: exited normally.
  (session not created: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
    #0 0x5c1c61f64fb3 <unknown>

System configuration

Rails version: 7.1.1

Ruby version: 3.0.6

[paulreece]

Hey there, thanks for bringing this to our attention. It seems like we do elaborate on this a bit in changing-the-default-settings.

By default Rails creates the new app with :

driven_by :selenium, using: :chrome, screen_size: [1400, 1400]

So locally if you have chrome installed it should work with both :chrome and :headless_chrome( it does for me locally).

We also mention this in the Docs I linked to:

If you want to use a remote browser, e.g. Headless Chrome in Docker, you have to add remote url and set browser as remote through options.

require "test_helper"

class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
  url = ENV.fetch("SELENIUM_REMOTE_URL", nil)
  options = if url
    { browser: :remote, url: url }
  else
    { browser: :chrome }
  end
  driven_by :selenium, using: :headless_chrome, options: options
end

So we do give a link to docker selenium as well as an example script for using headless_chrome in docker. Do you feel that we need to elaborate on this more?

[runephilosof-abtion]

If you want to use a remote browser, e.g. Headless Chrome in Docker, you have to add remote url and set browser as remote through options.

It is for a different use-case: For running headless Chrome in a different machine/VM/container. I am trying to run headless Chrome in the same container as Rails is running.

Do you feel that we need to elaborate on this more?

I don't think that would help, since it is a different use-case. However, it might.

The docs should mention that you must have the configured browsers installed.

In the case of Chrome it seems some additional configuration might be needed as well, but maybe that is not for the docs to describe. Maybe it is a bug to be fixed.

I have provided a complete reproducible example in the initial description. It only requires Docker.

[paulreece]

Thanks for clarifying!

To further troubleshoot, are you working on a Mac M1 or Mac M2 processor?

[runephilosof-abtion]

No, I am using Fedora / Gnu Linux on an x64 Intel processor.

[paulreece]

I've confirmed this. I am currently looking into what the best config option will be to get this running effectively but I do agree with you that this should be updated here if we can put a conditional for the newly added config options for when running in Docker: https://github.com/rails/rails/blob/02fa5c228892c0c5e109822cf09a1072827ee05e/actionpack/lib/action_dispatch/system_testing/browser.rb#L61-L66

[runephilosof-abtion]

I forgot to add a -i to the sed command in the description. I have added it.

[runephilosof-abtion]

When starting chromium --headless in the terminal, it says

root@9ac8c4414fe7:/test-system# chromium --headless
find: ‘/root/.config/chromium/Crash Reports/pending/’: No such file or directory
[1101/180813.656737:ERROR:zygote_host_impl_linux.cc(100)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.

I have tried to

adduser test
chown -R .
su test

But it still fails

test@9ac8c4414fe7:/test-system$ chromium --headless
find: ‘/home/test/.config/chromium/Crash Reports/pending/’: No such file or directory
Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted
[1101/181107.983538:FATAL:zygote_host_impl_linux.cc(200)] Check failed: . : Operation not permitted (1)
Trace/breakpoint trap (core dumped)

However, adding --no-sandbox in actionpack makes it work for both root and test users. https://github.com/rails/rails/blob/02fa5c228892c0c5e109822cf09a1072827ee05e/actionpack/lib/action_dispatch/system_testing/browser.rb#L61-L66

[rails-bot[bot]]

This issue has been automatically marked as stale because it has not been commented on for at least three months. The resources of the Rails team are limited, and so we are asking for your help. If you can still reproduce this error on the 7-1-stable branch or on main, please reply with all of the information you have about it in order to keep the issue open. Thank you for all your contributions.

[viktorianer]

Take a look at https://github.com/rails/rails/pull/50914. Actually, the guide needs an update here now.