geckodriver icon indicating copy to clipboard operation
geckodriver copied to clipboard

Page is not interactive until FireFox window is overlapped by another windows

Open ilya-corp opened this issue 1 year ago • 14 comments

System

  • Version: 10
  • Platform: Windows x64
  • Firefox: 103.0.2
  • Selenium: Geckodriver 0.31.0

Testcase

I have Selenium test, which perform opening of 3 instances of the browser and manipulating with elements. My problem is the following: if FireFox window is overlapped by another application (for example, MS Word is opened and overlap browsers), tested app does not working properly - it is not response for network events. BUT if I move FireFox to the front OR if I move mouse to Windows taskbar to display application mini-image, Firefox will work as expected. If I run my Selenium tests in headless mode, I also have NOT this problem. But when I run test from debugger, I need to move mouse to taskbar to kick browser. I have no this problem in Chrome.

It looks like some kind of optimization, but we should not use them during Selenium tests.

ilya-corp avatar Aug 22 '22 14:08 ilya-corp

I have the exact same issue.

CasperHooft avatar Aug 24 '22 08:08 CasperHooft

Could you please attach a trace-level log from geckodriver for the problematic Firefox instance? That might help to get further information. So far it's not something that I have seen during all the time I work on our WebDriver implementation. Maybe you also have a code snippet for reproduction?

whimboo avatar Sep 07 '22 07:09 whimboo

Could you please attach a trace-level log from geckodriver for the problematic Firefox instance? That might help to get further information. So far it's not something that I have seen during all the time I work on our WebDriver implementation. Maybe you also have a code snippet for reproduction?

trace-log-geckodriver.txt Here is the trace-log, if have exact the same problem.

Gerald94 avatar Sep 09 '22 07:09 Gerald94

@Gerald94 can you please point to a specific line of code which fails including an element reference (id, xpath, css class name or whatever)? This would help me a lot.

Note that in your case a lot of Javascript gets executed including clicks on elements. So there is the potential risk that the framework that you are using might have a bug. There is clearly the WebDriver:ElementClick command that should be used instead.

whimboo avatar Sep 09 '22 07:09 whimboo

@Gerald94 can you please point to a specific line of code which fails including an element reference (id, xpath, css class name or whatever)? This would help me a lot.

Note that in your case a lot of Javascript gets executed including clicks on elements. So there is the potential risk that the framework that you are using might have a bug. There is clearly the WebDriver:ElementClick command that should be used instead.

Yes sure, the test always fails on the same code line: "WebDriver:FindElements",{"using":"xpath","value":"//content-stammdaten-card"}]

If the Firefox Window is in foreground, the test is green.

Gerald94 avatar Sep 09 '22 08:09 Gerald94

Out of interest which kind of element is that? Is it a popup / overlay or one that gets lazily loaded via AJAX? Could you maybe create a minimized testcase to reproduce it?

whimboo avatar Sep 09 '22 09:09 whimboo

Out of interest which kind of element is that? Is it a popup / overlay or one that gets lazily loaded via AJAX? Could you maybe create a minimized testcase to reproduce it?

It is an normal div element, i tried to click other elements, but also got the same error, so it is hard to reproduce. For me it seems that the geckodriver loses the connection after some time when the browser is placed in the background.

Gerald94 avatar Sep 09 '22 09:09 Gerald94

That's not the case, at least for the trace log that you uploaded. The element with the xpath of content-stammdaten-card is not found in the page and then your code retries the same step for about 2s before a screenshot is taken and the session be deleted. So the question is what happens when you extend this timeout to eg. 10s and after 5s making the Firefox window visible?

whimboo avatar Sep 09 '22 11:09 whimboo

That's not the case, at least for the trace log that you uploaded. The element with the xpath of content-stammdaten-card is not found in the page and then your code retries the same step for about 2s before a screenshot is taken and the session be deleted. So the question is what happens when you extend this timeout to eg. 10s and after 5s making the Firefox window visible?

So now i have set the timeout to 10s, the element is even not found, after 5s i open the firefox window, then he finds the element.

Gerald94 avatar Sep 09 '22 11:09 Gerald94

I can confirm this. I'm working on anonymizing the Trace Logs. But in the Logs it looks just like the element is simply not in the Viewport. As soon as Firefox is in the foreground (Really meaning Visual foreground in the window context) element can be scrolled into view and clicked.

moesfeld avatar Sep 21 '22 12:09 moesfeld

This is the Trace Log with obscured window (Fail) TraceWindowObscured.txt This is the Trace Log with unobscured window (Success) TraceWindowNotObscured.txt

I will try to pinpoint the Version this behaviour was introduced. Might even be depending on the Firefox Version. I'm already using WebDriver:ElementClick. I wonder why Chrome is handling it without Issues

Edit: OS Version: Windows 10 Platform: Windows x64 Firefox: 104.0.2 Geckodriver 0.31.0 Selenium 4.4.0

moesfeld avatar Sep 22 '22 08:09 moesfeld

Please note that the last trace log with the window obscured shows a different failure. It's not that the NotesArticleIcon element cannot be found but it's not reachable for issuing a click:

1663831777969	webdriver::server	DEBUG	<- 400 Bad Request {"value":{"error":"element not interactable","message":"Element <div class=\"NotesArticleIcon js-initialized\"> could not be scrolled into view","stacktrace":"RemoteError@chrome://remote/content/shared/RemoteError.jsm:12:1\nWebDriverError@chrome://remote/content/shared/webdriver/Errors.jsm:192:5\nElementNotInteractableError@chrome://remote/content/shared/webdriver/Errors.jsm:302:5\nwebdriverClickElement@chrome://remote/content/marionette/interaction.js:156:11\ninteraction.clickElement@chrome://remote/content/marionette/interaction.js:125:11\nclickElement@chrome://remote/content/marionette/actors/MarionetteCommandsChild.jsm:204:29\nreceiveMessage@chrome://remote/content/marionette/actors/MarionetteCommandsChild.jsm:92:31\n"}}

Is that a regression? If yes, in which version of Firefox did this problem exactly start? I tried locally but I cannot reproduce it yet. So I would appreciate a minimized code and test example.

whimboo avatar Sep 22 '22 18:09 whimboo

Hi, seems to be a regression. I'm working on a complete code sample but it's hard to pinpoint a certain combination. For now it seems to be codependent on geckodriver and Firefox version. I know hard facts are needed but I got a feeling that it's more up to Firefox then geckodriver. I'll get back on this after the weekend I guess.

moesfeld avatar Sep 22 '22 21:09 moesfeld

Thanks! It would be helpful to not only get the regression range for the release but maybe a check if the beta 1 release of that version is also affected. Can you reproduce your problem locally? If yes mozregression is a great help in combination with the -c argument. Then you can use the MOZREGRESSION_BINARY environment variable to specify the version of Firefox in your test.

whimboo avatar Sep 23 '22 06:09 whimboo

It is locally reproducible. But I'm having a bit of trouble with mozregression. Seems env is only set when using automated --command argument. Since I have to do reproduction manually (overlapping window, test case always returning 0...) I'd rather be using the manual mode or even mozregression GUI. But I will figure it out eventually. Just takes a bit more Time.

moesfeld avatar Sep 27 '22 06:09 moesfeld

Wouldn't it help to add some delays in the test code so that the test doesn't start immediately? Would that give you enough time to prepare Firefox for this situation?

whimboo avatar Sep 27 '22 07:09 whimboo

Figured it out how to automate it and got some first results. Nifty tool! I'll try to dig deeper:

9:02.06 INFO: Test command result: 0 (build is good) 9:02.06 INFO: Narrowed integration regression window from [f3da8920, 4dc0e294] (3 builds) to [a4739fdb, 4dc0e294] (2 builds) (~1 steps left) 9:02.06 INFO: No more integration revisions, bisection finished. 9:02.06 INFO: Last good revision: a4739fdb1d1cffb06d5971a6fb6901c1b3b41d26 9:02.06 INFO: First bad revision: 4dc0e294453b58a010a1a78504fdbd8488c57a37 9:02.06 INFO: Pushlog: https://hg.mozilla.org/releases/mozilla-release/pushloghtml?fromchange=a4739fdb1d1cffb06d5971a6fb6901c1b3b41d26&tochange=4dc0e294453b58a010a1a78504fdbd8488c57a37

moesfeld avatar Sep 29 '22 14:09 moesfeld

Hm, that's interesting. With a quick look it seems to be something that has been changed in Firefox 98. I'm curious to see a more fine-graned result given that there are a lot of commits left over to test for mozregression. Thanks for doing that!

whimboo avatar Sep 30 '22 12:09 whimboo

I checked the Nightlies: 10:53.04 INFO: Test command result: 1 (build is bad) 10:53.04 INFO: Narrowed integration regression window from [adb05036, 442a7912] (3 builds) to [adb05036, a297bc48] (2 builds) (~1 steps left) 10:53.04 INFO: No more integration revisions, bisection finished. 10:53.04 INFO: Last good revision: adb050364e8646f1a9efa890a5aaa7d71e6c3c3b 10:53.04 INFO: First bad revision: a297bc48dfeb404fcb9ada705d89d67e81cfd7bb 10:53.04 INFO: Pushlog: https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=adb050364e8646f1a9efa890a5aaa7d71e6c3c3b&tochange=a297bc48dfeb404fcb9ada705d89d67e81cfd7bb

The only commit was Bug 1733423 But this is not inside the first bisection. Wierd! I wonder if I had false positives in the first run

moesfeld avatar Sep 30 '22 12:09 moesfeld

Bug 1733423 doesn't really make sense given that this is for Android only.

whimboo avatar Sep 30 '22 12:09 whimboo

Thats what i thougt. But i checked the Logs and in the regression window there are some Builds that don't even respond to geckodriver at all. So some changes might have been skipped because they were considered to be "on the dark side" already. I will have look at that.

moesfeld avatar Sep 30 '22 12:09 moesfeld

Sorry, but I think you can omit what I wrote. I have to rethink my testcase. There are to many factors in it that create false negatives. I will have another go when I sorted that out.

moesfeld avatar Sep 30 '22 12:09 moesfeld

I encountered this issue today and was wondering if there was any movement on this.

I can confirm that it works on Firefox 97.0.2 but not in Firefox 98.

Like OP said, it's fine if I bring it to the foreground or if I hover over the window in the taskbar.

brycetham avatar Oct 21 '22 20:10 brycetham

Same problem here.

But I guess you guys dig in a wrong direction - seems the issue is with FF starter to throttle rendering of the backgroung windows/tabs. So it seems the issue is not about a geckodriver but in a way to tell FF we want to disable all the throttling.

We run about 8 instances of a FF simultaneously and face this issue regularly

ivorobkalo avatar Oct 25 '22 12:10 ivorobkalo

Sorry for the long wait. I was on vacation for a couple of weeks. You might be onto something here. Firefox 96 introduced throttling of occluded windows. It might be as easy as setting a config parameter. I will check that.

moesfeld avatar Oct 25 '22 14:10 moesfeld

@moesfeld I've tried different config settings but found no way to disable throttling. If you find those "magic" config setting - please let others know

ivorobkalo avatar Oct 26 '22 07:10 ivorobkalo

@ivorobkalo I can confirm setting widget.windows.window_occlusion_tracking.enabled to false did the trick for me. Firefox 106.1 Geckodriver 0.31.0 Win 10 Selenium 4.4.0

moesfeld avatar Oct 26 '22 10:10 moesfeld

@ivorobkalo I can confirm setting widget.windows.window_occlusion_tracking.enabled to false did the trick for me.

Oh neat this works! Thanks for looking into it.

brycetham avatar Oct 27 '22 16:10 brycetham

@ivorobkalo I can confirm setting widget.windows.window_occlusion_tracking.enabled to false did the trick for me. Firefox 106.1 Geckodriver 0.31.0 Win 10 Selenium 4.4.0

Thank you very much, it also works for me now!

Gerald94 avatar Oct 27 '22 16:10 Gerald94

@ivorobkalo I can confirm setting widget.windows.window_occlusion_tracking.enabled to false did the trick for me. Firefox 106.1 Geckodriver 0.31.0 Win 10 Selenium 4.4.0

Thanks a lot!

ivorobkalo avatar Nov 03 '22 14:11 ivorobkalo