selenium icon indicating copy to clipboard operation
selenium copied to clipboard

[🐛 Bug]: SessionNotCreatedException on GitHub Actions on Ubuntu 24.04

Open madebr opened this issue 1 year ago • 1 comments

What happened?

We (=the libsdl-org/SDL project) are using Selenium to run the emscripten tests in ci. When bumping the os from ubuntu-22.04 to ubuntu-24.04, all tests fail.

Link to a "good log". Link to a "bad log"

How can we reproduce the issue?

This only reproduces on GitHub Actions.


# Clone SDL repo locally
cd /tmp
git clone https://github.com/madebr/SDL
cd SDL
git checkout emscripten-selenium-ubuntu-24-04

# Create new GitHub repo
gh repo create --public <your-user>/SDL-test

# Add the just-created GitHub repo as a new remote
git remote add origin [email protected]:<your-user>/SDL-test

# Push SDL to your new GitHub repo
git push --set-upstream origin emscripten-selenium-ubuntu-24-04:refs/heads/emscripten-selenium-ubuntu-24-04

Visit https://github.com//SDL-test/actions in your browser to see the output.

Relevant log output

1: Traceback (most recent call last):
1:   File "/home/runner/work/SDL/SDL/test/emscripten/driver.py", line 183, in <module>
1:     raise SystemExit(main())
1:                      ^^^^^^
1:   File "/home/runner/work/SDL/SDL/test/emscripten/driver.py", line 160, in main
1:     sdl_test_driver = SDLSeleniumTestDriver(
1:                       ^^^^^^^^^^^^^^^^^^^^^^
1:   File "/home/runner/work/SDL/SDL/test/emscripten/driver.py", line 56, in __init__
1:     self.driver = driver_contructor(options=driver_options)
1:                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1:   File "/home/runner/.local/lib/python3.12/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
1:     super().__init__(
1:   File "/home/runner/.local/lib/python3.12/site-packages/selenium/webdriver/chromium/webdriver.py", line 66, in __init__
1:     super().__init__(command_executor=executor, options=options)
1:   File "/home/runner/.local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 212, in __init__
1:     self.start_session(capabilities)
1:   File "/home/runner/.local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 299, in start_session
1:     response = self.execute(Command.NEW_SESSION, caps)["value"]
1:                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1:   File "/home/runner/.local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 354, in execute
1:     self.error_handler.check_response(response)
1:   File "/home/runner/.local/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
1:     raise exception_class(message, screen, stacktrace)
1: selenium.common.exceptions.SessionNotCreatedException: Message: session not created: Chrome failed to start: exited normally.
1:   (session not created: DevToolsActivePort file doesn't exist)
1:   (The process started from chrome location /opt/hostedtoolcache/setup-chrome/chromium/1369611/x64/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
1: Stacktrace:
1: #0 0x5646e1b69cd2 <unknown>
1: #1 0x5646e1b5ba5e <unknown>
1: #2 0x5646e1607f27 <unknown>
1: #3 0x5646e163dda4 <unknown>
1: #4 0x5646e16381a6 <unknown>
1: #5 0x5646e168335d <unknown>
1: #6 0x5646e1682ad6 <unknown>
1: #7 0x5646e1675fe3 <unknown>
1: #8 0x5646e1645c41 <unknown>
1: #9 0x5646e1644d39 <unknown>
1: #10 0x5646e1645a1c <unknown>
1: #11 0x5646e1b1729f <unknown>
1: #12 0x5646e1b323d3 <unknown>
1: #13 0x5646e1b31e3b <unknown>
1: #14 0x5646e1b32845 <unknown>
1: #15 0x5646e1b21ab3 <unknown>
1: #16 0x5646e1b32be0 <unknown>
1: #17 0x5646e1b07ee1 <unknown>
1: #18 0x5646e1b4c248 <unknown>
1: #19 0x5646e1b4c3d9 <unknown>
1: #20 0x5646e1b5ad65 <unknown>
1: #21 0x7f01d389ca94 <unknown>
1: #22 0x7f01d3929c3c <unknown>

Operating System

Ubuntu 24.04

Selenium version

Python, 4.25.0

What are the browser(s) and version(s) where you see this issue?

Chrome 132.0.6781.0

What are the browser driver(s) and version(s) where you see this issue?

ChromeDrive 132.0.6781.0

Are you using Selenium Grid?

n/a

madebr avatar Oct 16 '24 22:10 madebr

@madebr, thank you for creating this issue. We will troubleshoot it as soon as we can.


Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

github-actions[bot] avatar Oct 16 '24 22:10 github-actions[bot]

What if option --headless=new ?

VietND96 avatar Nov 20 '24 04:11 VietND96

I bumped the emscripten job to ubuntu-24.04 and selenium fails to "obtain a working Selenium Manager binary".

Unable to obtain working Selenium Manager binary

https://github.com/madebr/SDL/actions/runs/11935153665/job/33265980644#step:25:46

2: Traceback (most recent call last):
2:   File "/usr/lib/python3/dist-packages/selenium/webdriver/common/driver_finder.py", line 38, in get_path
2:     path = SeleniumManager().driver_location(options) if path is None else path
2:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2:   File "/usr/lib/python3/dist-packages/selenium/webdriver/common/selenium_manager.py", line 89, in driver_location
2:     args = [str(self.get_binary()), "--browser", browser]
2:                 ^^^^^^^^^^^^^^^^^
2:   File "/usr/lib/python3/dist-packages/selenium/webdriver/common/selenium_manager.py", line 73, in get_binary
2:     raise WebDriverException(f"Unable to obtain working Selenium Manager binary; {path}")
2: selenium.common.exceptions.WebDriverException: Message: Unable to obtain working Selenium Manager binary; /usr/lib/python3/dist-packages/selenium/webdriver/common/linux/selenium-manager
2: 
2: 
2: The above exception was the direct cause of the following exception:
2: 
2: Traceback (most recent call last):
2:   File "/home/runner/work/SDL/SDL/test/emscripten/driver.py", line 183, in <module>
2:     raise SystemExit(main())
2:                      ^^^^^^
2:   File "/home/runner/work/SDL/SDL/test/emscripten/driver.py", line 160, in main
2:     sdl_test_driver = SDLSeleniumTestDriver(
2:                       ^^^^^^^^^^^^^^^^^^^^^^
2:   File "/home/runner/work/SDL/SDL/test/emscripten/driver.py", line 56, in __init__
2:     self.driver = driver_contructor(options=driver_options)
2:                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2:   File "/usr/lib/python3/dist-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
2:     super().__init__(
2:   File "/usr/lib/python3/dist-packages/selenium/webdriver/chromium/webdriver.py", line 49, in __init__
2:     self.service.path = DriverFinder.get_path(self.service, options)
2:                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2:   File "/usr/lib/python3/dist-packages/selenium/webdriver/common/driver_finder.py", line 41, in get_path
2:     raise NoSuchDriverException(msg) from err
2: selenium.common.exceptions.NoSuchDriverException: Message: Unable to obtain driver for chrome using Selenium Manager.; For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors/driver_location
2: 

I'm using python3-selenium from Ubuntu's apt-get.

madebr avatar Nov 20 '24 14:11 madebr

@bonigarcia Is this a known issue? Any suggestions from your end? Thank you!

pujagani avatar Dec 13 '24 08:12 pujagani

There are similar issues to this, and they are related to the use of a custom Python package manager. For example:

https://github.com/SeleniumHQ/selenium/issues/14488

See this comment:

https://github.com/SeleniumHQ/selenium/issues/14488#issuecomment-2344274646

bonigarcia avatar Dec 13 '24 08:12 bonigarcia

Using selenium from pypi also fails. The installation happens here and the test fails here.

Log extract
10: Test command: /usr/bin/python3 "/home/runner/work/SDL/SDL/test/emscripten/driver.py" "--server" "http://localhost:8080" "--browser" "chrome" "--chrome-binary" "/opt/hostedtoolcache/setup-chrome/chromium/1396600/x64/chrome" "--" "testplatform" "--trackmem"
10: Working Directory: /home/runner/work/SDL/SDL/build/test
10: Environment variables: 
10:  SDL_AUDIO_DRIVER=emscripten
10:  SDL_VIDEO_DRIVER=emscripten
10:  SDL_ASSERT=abort
10: Test timeout computed to be: 40
10: Traceback (most recent call last):
10:   File "/home/runner/work/SDL/SDL/test/emscripten/driver.py", line 183, in <module>
10:     raise SystemExit(main())
10:                      ^^^^^^
10:   File "/home/runner/work/SDL/SDL/test/emscripten/driver.py", line 160, in main
10:     sdl_test_driver = SDLSeleniumTestDriver(
10:                       ^^^^^^^^^^^^^^^^^^^^^^
10:   File "/home/runner/work/SDL/SDL/test/emscripten/driver.py", line 56, in __init__
10:     self.driver = driver_contructor(options=driver_options)
10:                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
10:   File "/home/runner/.local/lib/python3.12/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
10:     super().__init__(
10:   File "/home/runner/.local/lib/python3.12/site-packages/selenium/webdriver/chromium/webdriver.py", line 66, in __init__
10:     super().__init__(command_executor=executor, options=options)
10:   File "/home/runner/.local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 241, in __init__
10:     self.start_session(capabilities)
10:   File "/home/runner/.local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 329, in start_session
10:     response = self.execute(Command.NEW_SESSION, caps)["value"]
10:                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
10:   File "/home/runner/.local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 384, in execute
10:     self.error_handler.check_response(response)
10:   File "/home/runner/.local/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 232, in check_response
10:     raise exception_class(message, screen, stacktrace)
10: selenium.common.exceptions.SessionNotCreatedException: Message: session not created: probably user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir

(We don't use --user-data-dir)

madebr avatar Dec 16 '24 09:12 madebr

Also hitting it:

Droplet Instance Ubuntu 24.10 Image: s-2vcpu-4gb, but any will probably work with enough ram

Install:

apt install -y python3 python3-pip
pip install uv --break-system-packages
uv venv
uv pip install seleniumbase

Then I paste this script in

from seleniumbase import BaseCase
BaseCase.main(__name__, __file__)  # Call pytest

class MyTestClass(BaseCase):
    def test_swag_labs(self):
        self.open("https://www.saucedemo.com")
        self.type("#user-name", "standard_user")
        self.type("#password", "secret_sauce\n")
        self.assert_element("div.inventory_list")
        self.click('button[name*="backpack"]')
        self.click("#shopping_cart_container a")
        self.assert_text("Backpack", "div.cart_item")
        self.click("button#checkout")
        self.type("input#first-name", "SeleniumBase")
        self.type("input#last-name", "Automation")
        self.type("input#postal-code", "77123")
        self.click("input#continue")
        self.click("button#finish")
        self.assert_text("Thank you for your order!")

This is the output I get back:

============================================ {  File "<stdin>:1:SB} starts =============================================

Warning: chromedriver not found. Getting it now:

*** chromedriver to download = 132.0.6834.83 (Latest Stable) 

Downloading chromedriver-linux64.zip from:
https://storage.googleapis.com/chrome-for-testing-public/132.0.6834.83/linux64/chromedriver-linux64.zip ...
Download Complete!

Extracting ['chromedriver'] from chromedriver-linux64.zip ...
Unzip Complete!

The file [chromedriver] was saved to:
/root/.venv/lib/python3.12/site-packages/seleniumbase/drivers/
chromedriver

Making [chromedriver 132.0.6834.83] executable ...
[chromedriver 132.0.6834.83] is now ready for use!

Traceback (most recent call last):
  File "/root/.venv/lib/python3.12/site-packages/seleniumbase/core/browser_launcher.py", line 5223, in get_local_driver
    driver = webdriver.Chrome(options=chrome_options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
    super().__init__(
  File "/root/.venv/lib/python3.12/site-packages/selenium/webdriver/chromium/webdriver.py", line 66, in __init__
    super().__init__(command_executor=executor, options=options)
  File "/root/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 241, in __init__
    self.start_session(capabilities)
  File "/root/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 329, in start_session
    response = self.execute(Command.NEW_SESSION, caps)["value"]
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 384, in execute
    self.error_handler.check_response(response)
  File "/root/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 232, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not created: probably user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir
Stacktrace:
#0 0x65018ac177ca <unknown>
#1 0x65018a70f2f0 <unknown>
#2 0x65018a746063 <unknown>
#3 0x65018a7428c6 <unknown>
#4 0x65018a78fbc9 <unknown>
#5 0x65018a78f216 <unknown>
#6 0x65018a783753 <unknown>
#7 0x65018a750baa <unknown>
#8 0x65018a751dfe <unknown>
#9 0x65018abe238b <unknown>
#10 0x65018abe6307 <unknown>
#11 0x65018abcee7c <unknown>
#12 0x65018abe6ec7 <unknown>
#13 0x65018abb324f <unknown>
#14 0x65018ac062f8 <unknown>
#15 0x65018ac064c0 <unknown>
#16 0x65018ac16646 <unknown>
#17 0x76778b6a1e2e <unknown>
#18 0x76778b733a4c <unknown>


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/seleniumbase/plugins/sb_manager.py", line 1214, in SB
    sb.setUp()
  File "/root/.venv/lib/python3.12/site-packages/seleniumbase/fixtures/base_case.py", line 15191, in setUp
    self.driver = self.get_new_driver(
                  ^^^^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/seleniumbase/fixtures/base_case.py", line 4183, in get_new_driver
    new_driver = browser_launcher.get_driver(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/seleniumbase/core/browser_launcher.py", line 2845, in get_driver
    return get_local_driver(
           ^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/seleniumbase/core/browser_launcher.py", line 5304, in get_local_driver
    driver = webdriver.Chrome(
             ^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
    super().__init__(
  File "/root/.venv/lib/python3.12/site-packages/selenium/webdriver/chromium/webdriver.py", line 66, in __init__
    super().__init__(command_executor=executor, options=options)
  File "/root/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 241, in __init__
    self.start_session(capabilities)
  File "/root/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 329, in start_session
    response = self.execute(Command.NEW_SESSION, caps)["value"]
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 384, in execute
    self.error_handler.check_response(response)
  File "/root/.venv/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 232, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not created: probably user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir
Stacktrace:
#0 0x60e74374c7ca <unknown>
#1 0x60e7432442f0 <unknown>
#2 0x60e74327b063 <unknown>
#3 0x60e7432778c6 <unknown>
#4 0x60e7432c4bc9 <unknown>
#5 0x60e7432c4216 <unknown>
#6 0x60e7432b8753 <unknown>
#7 0x60e743285baa <unknown>
#8 0x60e743286dfe <unknown>
#9 0x60e74371738b <unknown>
#10 0x60e74371b307 <unknown>
#11 0x60e743703e7c <unknown>
#12 0x60e74371bec7 <unknown>
#13 0x60e7436e824f <unknown>
#14 0x60e74373b2f8 <unknown>
#15 0x60e74373b4c0 <unknown>
#16 0x60e74374b646 <unknown>
#17 0x71c0ae6a1e2e <unknown>
#18 0x71c0ae733a4c <unknown>

Most likely this can be fixed with a helpful error message. It's on a worker machine and remote debugging isn't easy.

Can anyone help me figure out a work around? I'm guessing it's probably the binary failing to start. Process failure message seems to be swallowed up.

Devs, can you insert a warning with whatever subprocess.run was streaming?

zackees avatar Jan 15 '25 22:01 zackees

Are you on Ubuntu 24? There's a known compatibility issue. Downgrade to Ubuntu 22

zackees avatar Jan 16 '25 00:01 zackees

Are you on Ubuntu 24? There's a known compatibility issue. Downgrade to Ubuntu 22

That's what we (=libsdl-org/SDL) are doing currently. Do you perhaps have a link to an issue that I can upvote/follow?

madebr avatar Jan 16 '25 01:01 madebr

Recently, GitHub updated ubuntu-latest from 22.04 to 24.04. In this update, it looks like Chrome/Edge didn't wrap with --no-sandbox by default anymore. Which causes the error

selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir

Refer to an explanation https://stackoverflow.com/questions/70385737/selenium-common-exceptions-invalidargumentexception-message-invalid-argument

Update ChromeOptions/EdgeOptions always append argument --no-sandbox when creating WebDriver looks like a solution till now. An update to prove it works can be referred to https://github.com/SeleniumHQ/seleniumhq.github.io/pull/2139

VietND96 avatar Jan 21 '25 08:01 VietND96

I'm using the pypi selenium package. When can I expect this change to become available?

madebr avatar Jan 21 '25 20:01 madebr

I'm using the pypi selenium package. When can I expect this change to become available?

Actually, this change can be done on your end. If you are using Python Selenium, I think your code where WebDriver is created looks like

    options = webdriver.ChromeOptions()
    options.add_argument("--no-sandbox")
    driver = webdriver.Chrome(options=options)

VietND96 avatar Jan 21 '25 23:01 VietND96

Yup, that makes ci complete!

Thank you very much for the help! I'm leaving the honors of closing this issue to you :)

madebr avatar Jan 22 '25 01:01 madebr

Image

hello, although the project working but I got an error.Why the reason for error?

Feyzaaaa avatar Jan 25 '25 16:01 Feyzaaaa

I am using Firefox. I tried the headless mode for both Ubuntu 22.04 and 24.04 base images. None of them worked.

leimao avatar Feb 05 '25 07:02 leimao

I tried Chrome in Ubuntu 22.04 base image. Surprisingly it worked. I have no idea what happened and why Firefox did not work.

leimao avatar Feb 05 '25 07:02 leimao

@leimao, the origin issue was raised with Chrome and Ubuntu 24.04. We are happy to help you troubleshoot further if you can create a new ticket with as many details as possible and a script to reproduce the issue.

VietND96 avatar Feb 05 '25 12:02 VietND96

This issue has been automatically locked since there has not been any recent activity since it was closed. Please open a new issue for related bugs.

github-actions[bot] avatar Mar 07 '25 22:03 github-actions[bot]