PR for regression testing for https://github.com/mozilla/geckodriver/issues/2042.

Sep 13 '22 15:09 whimboo

Codecov Report

Base: 53.18% // Head: 53.18% // No change to project coverage :thumbsup:

Coverage data is based on head (8d115d9) compared to base (b2598e7). Patch has no changes to coverable lines.

Additional details and impacted files

@@           Coverage Diff           @@
##            trunk   #11025   +/-   ##
=======================================
  Coverage   53.18%   53.18%           
=======================================
  Files          84       84           
  Lines        5488     5488           
  Branches      272      272           
=======================================
  Hits         2919     2919           
  Misses       2297     2297           
  Partials      272      272

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

Sep 13 '22 15:09 codecov-commenter

sounds like we found the culprit!

Sep 14 '22 14:09 titusfortner

With the latest push the failure seem to be fixed. We are in process to decide which patches exactly need to be backed out. So once done and a Nightly build shipped I would like to push one more time to this PR for verification.

It's really interesting that this issue can only be seen in GitHub Actions.

Sep 14 '22 20:09 whimboo

@titusfortner I could actually need your help here for the failure in the Ruby bindings which is related to the assumption that a window manager would set the DESKTOP_SESSION. Like the following tests work now with a window manager running:

  1) Selenium::WebDriver::Window can maximize the current window FIXED
     Expected pending 'Test guarded; no reason given' to fail. No error was raised.
     # ./rb/spec/integration/selenium/webdriver/window_spec.rb:108

  2) Selenium::WebDriver::Window can make window full screen FIXED
     Expected pending 'Test guarded; no reason given' to fail. No error was raised.
     # ./rb/spec/integration/selenium/webdriver/window_spec.rb:119

  3) Selenium::WebDriver::Window can minimize the window FIXED
     Expected pending 'Test guarded; no reason given' to fail. No error was raised.
     # ./rb/spec/integration/selenium/webdriver/window_spec.rb:130

I wonder if we should simply set the DESKTOP_SESSION=fluxbox environment variable manually given that Fluxbox isn't doing that, or removing this particular behavior completely and always enforce that a window manager is needed to run the browser tests. Without one all the window manipulation tests aren't guaranteed to work.

Regarding the Java failures I would suggest we defer until Ruby is done. Thanks

Sep 20 '22 06:09 whimboo

I wonder if we should simply set the DESKTOP_SESSION=fluxbox environment variable manually given that Fluxbox isn't doing that, or removing this particular behavior completely and always enforce that a window manager is needed to run the browser tests. Without one all the window manipulation tests aren't guaranteed to work.

If we need a way to detect a window manager a better way should be the following:

xprop -root -display :99

If it's not failing to connect to the display and no window manager is installed we should expect a couple of _NET_* lines to be returned.

Sep 20 '22 06:09 whimboo

No, this is great. We only guarded them that way because I want to be able to run locally on linux and remotely on travis/github and have everything "pass" for the right reasons. If everything guarded for window_manager now works, we can completely remove that guard.

Sep 20 '22 14:09 titusfortner

There is one Java test that now works with Firefox 105. I've created a separate PR: https://github.com/SeleniumHQ/selenium/pull/11046.

Sep 20 '22 18:09 whimboo

There are quite a few failures for Java on trunk as well. Those started between September 13th and 15th: https://github.com/SeleniumHQ/selenium/actions/runs/3092869938/jobs/5004771191

The failures look similar to me so I don't think that this PR should be blocked on them. @titusfortner if you could have a look and review this PR I would appreciate.

Sep 21 '22 19:09 whimboo

Kudos, SonarCloud Quality Gate passed!