docker-selenium icon indicating copy to clipboard operation
docker-selenium copied to clipboard

Published 20 hours ago • verified publisher icon SeleniumHQ

[🐛 Bug]: org.openqa.selenium.json.JsonException: Expected to read a START_MAP but instead have: END. Last 0 characters read:

Open mghantous opened this issue 3 years ago • 30 comments

What happened?

We have a python / django test suite that runs about 60 tests in serial against a standalone-chrome and also against standalone-firefox. Usually somewhere in the middle of the run, but varying on the specific test, we encounter an exception that is org.openqa.selenium.json.JsonException: Expected to read a START_MAP but instead have: END. Last 0 characters read:. The stack trace is almost identical for firefox and chrome other than the System Info (IP and host name), so I have attached one in the log output. The selenium python package and standalone images we use are both on the latest available tags (selenium==4.1.0 and 4.1.2-20220208 respectively).

Command used to start Selenium Grid with Docker

We start via docker-compose with standalone firefox and chrome images. The relevant compose sections:

  firefox:
    image: selenium/standalone-firefox:${SELENIUM_TAG:-4.1.2-20220208}
    shm_size: 2gb
    environment:
      - START_XVFB=${START_XVFB:-false}

  chrome:
    image: selenium/standalone-chrome:${SELENIUM_TAG:-4.1.2-20220208}
    shm_size: 2gb
    environment:
      - START_XVFB=${START_XVFB:-false}
      # Cannot bind to ipv6 on docker for linux. See https://stackoverflow.com/a/57328161/2112497
      - JAVA_OPTS=-Dwebdriver.chrome.whitelistedIps=

Relevant log output

org.openqa.selenium.json.JsonException: Unable to parse:
	at org.openqa.selenium.json.Json.toType(Json.java:57)
	at org.openqa.selenium.json.Json.toType(Json.java:50)
	at org.openqa.selenium.grid.node.CustomLocatorHandler.execute(CustomLocatorHandler.java:113)
	at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
	at org.openqa.selenium.grid.node.Node.execute(Node.java:240)
	at org.openqa.selenium.grid.web.CombinedHandler.execute(CombinedHandler.java:59)
	at org.openqa.selenium.grid.web.RoutableHttpClientFactory$1.execute(RoutableHttpClientFactory.java:72)
	at org.openqa.selenium.grid.web.ReverseProxyHandler.execute(ReverseProxyHandler.java:92)
	at org.openqa.selenium.grid.router.HandleSession.execute(HandleSession.java:110)
	at org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)
	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
	at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
	at org.openqa.selenium.grid.router.Router.execute(Router.java:91)
	at org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)
	at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
	at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
	at org.openqa.selenium.remote.http.Route$NestedRoute.handle(Route.java:270)
	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
	at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
	at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
	at org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)
	at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
	at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
	at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
	at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
	at org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.openqa.selenium.json.JsonException: Expected to read a START_MAP but instead have: END. Last 0 characters read:
Build info: version: '4.1.2', revision: '9a5a329c5a'
System info: host: 'ecba5910c3b1', ip: '172.18.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.0-97-generic', java.version: '11.0.13'
Driver info: driver.version: unknown
	at org.openqa.selenium.json.JsonInput.expect(JsonInput.java:307)
	at org.openqa.selenium.json.JsonInput.beginObject(JsonInput.java:229)
	at org.openqa.selenium.json.MapCoercer.lambda$apply$1(MapCoercer.java:64)
	at org.openqa.selenium.json.JsonTypeCoercer.lambda$buildCoercer$6(JsonTypeCoercer.java:145)
	at org.openqa.selenium.json.JsonTypeCoercer.coerce(JsonTypeCoercer.java:127)
	at org.openqa.selenium.json.Json.toType(Json.java:71)
	at org.openqa.selenium.json.Json.toType(Json.java:55)
	... 35 more

Operating System

Ubuntu 20.04

Docker Selenium version (tag)

4.1.2-20220208

mghantous avatar Feb 09 '22 17:02 mghantous

How can I reproduce the issue?

diemol avatar Feb 17 '22 22:02 diemol

We're encountering the same thing. I have no reliable way to reproduce it though.

indrekj avatar Mar 04 '22 13:03 indrekj

If it helps then this happens with 4.1.0 (and 4.1.2) for us as well:

15:55:14       Selenium::WebDriver::Error::UnknownError:
15:55:14         Expected to read a START_MAP but instead have: END. Last 0 characters read: 
15:55:14         Build info: version: '4.1.0', revision: 'aea69daa1c'
15:55:14         System info: host: 'beta-monitoring-dff87366-8287-4ca3-bcfd-092b2526c15d-4dzb-wsx0j', ip: '10.2.127.116', os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.172-90.336.amzn2.x86_64', java.version: '11.0.11'
15:55:14         Driver info: driver.version: unknown

indrekj avatar Mar 04 '22 13:03 indrekj

I also encountered the issue. Test runs fine in 20 to 25 threads but running on 30 threads (or more) will result to about 50% of the tests getting the mentioned error. The higher the number of threads, the higher the percentage of the error.

chinzzz avatar Mar 21 '22 06:03 chinzzz

We also see this issue with selenium-server-4.1.2. The problem is not reproducible. But all failing tests have two things in common:

  1. The client-side stacktrace shows the following lines (java bindings 4.1.2):
at org.openqa.selenium.remote.TracedCommandExecutor.execute(TracedCommandExecutor.java:51)
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:558)
at org.openqa.selenium.remote.ElementLocation$ElementFinder$2.findElements(ElementLocation.java:179)
at org.openqa.selenium.remote.ElementLocation.findElements(ElementLocation.java:102)
  1. Another warning is logged before and/or after the log message containing the server-side exception. This warning is logged for the same session id and contains the http status code 404:
eventName: HTTP request execution complete
attributes: 
	http.flavor: 1
	http.handler_class: org.openqa.selenium.remote.http.Route$PredicatedRoute
	http.host: xxx
	http.method: POST
	http.request_content_length: yyy
	http.scheme: HTTP
	http.status_code: 404
	http.target: \u002fsession\u002fada9755d04e47dc8938ea8008b214324\u002fexecute\u002fsync
	http.user_agent: selenium\u002f4.1.2 (java unix)

thkwalter avatar Mar 21 '22 14:03 thkwalter

I've had a look at the comments but sadly I've got not clue on how to reproduce this. Is this still happening in 4.1.3?

diemol avatar Apr 08 '22 06:04 diemol

Yes, still happening on 4.1.3

chrome_1               | 2022-04-11T15:56:06.218687658Z 15:56:06.218 INFO [LocalDistributor.newSession] - Session created by the distributor. Id: 43e21e36cc481382130f3670adbf9b05, Caps: Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 100.0.4896.75, chrome: {chromedriverVersion: 100.0.4896.60 (6a5d10861ce8..., userDataDir: /tmp/.com.google.Chrome.DDSm8a}, goog:chromeOptions: {debuggerAddress: localhost:37813}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: linux, proxy: Proxy(), se:cdp: ws://172.18.0.4:4444/sessio..., se:cdpVersion: 100.0.4896.75, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:virtualAuthenticators: true}
chrome_1               | 2022-04-11T15:56:28.730011666Z 15:56:28.729 WARN [SeleniumSpanExporter$1.lambda$export$0] - {"traceId": "571a4e5092c697880a6857b42316f3b8","eventTime": 1649692588724185395,"eventName": "exception","attributes": {"exception.message": "Unable to execute request for an existing session: Unable to parse: ","exception.stacktrace": "org.openqa.selenium.json.JsonException: Unable to parse: \n\tat org.openqa.selenium.json.Json.toType(Json.java:57)\n\tat org.openqa.selenium.json.Json.toType(Json.java:50)\n\tat org.openqa.selenium.grid.node.CustomLocatorHandler.execute(CustomLocatorHandler.java:113)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.grid.node.Node.execute(Node.java:240)\n\tat org.openqa.selenium.grid.web.CombinedHandler.execute(CombinedHandler.java:59)\n\tat org.openqa.selenium.grid.web.RoutableHttpClientFactory$1.execute(RoutableHttpClientFactory.java:72)\n\tat org.openqa.selenium.grid.web.ReverseProxyHandler.execute(ReverseProxyHandler.java:92)\n\tat org.openqa.selenium.grid.router.HandleSession.execute(HandleSession.java:121)\n\tat org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.grid.router.Router.execute(Router.java:91)\n\tat org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$NestedRoute.handle(Route.java:270)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)\n\tat org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)\n\tat java.base\u002fjava.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base\u002fjava.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base\u002fjava.lang.Thread.run(Thread.java:829)\nCaused by: org.openqa.selenium.json.JsonException: Expected to read a START_MAP but instead have: END. Last 0 characters read: \nBuild info: version: '4.1.3', revision: '7b1ebf28ef'\nSystem info: host: '83ee97fd6958', ip: '172.18.0.4', os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.0-107-generic', java.version: '11.0.14.1'\nDriver info: driver.version: unknown\n\tat org.openqa.selenium.json.JsonInput.expect(JsonInput.java:307)\n\tat org.openqa.selenium.json.JsonInput.beginObject(JsonInput.java:229)\n\tat org.openqa.selenium.json.MapCoercer.lambda$apply$1(MapCoercer.java:64)\n\tat org.openqa.selenium.json.JsonTypeCoercer.lambda$buildCoercer$6(JsonTypeCoercer.java:145)\n\tat org.openqa.selenium.json.JsonTypeCoercer.coerce(JsonTypeCoercer.java:127)\n\tat org.openqa.selenium.json.Json.toType(Json.java:71)\n\tat org.openqa.selenium.json.Json.toType(Json.java:55)\n\t... 35 more\n","exception.type": "org.openqa.selenium.json.JsonException","http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.router.HandleSession","http.host": "chrome:4444","http.method": "POST","http.request_content_length": "60","http.scheme": "HTTP","http.target": "\u002fsession\u002f43e21e36cc481382130f3670adbf9b05\u002felement","http.user_agent": "selenium\u002f4.1.3 (python linux)","session.id": "43e21e36cc481382130f3670adbf9b05"}}

Let me try to format that stack trace multi-line

  org.openqa.selenium.json.JsonException: Unable to parse: 
	at org.openqa.selenium.json.Json.toType(Json.java:57)
	at org.openqa.selenium.json.Json.toType(Json.java:50)
	at org.openqa.selenium.grid.node.CustomLocatorHandler.execute(CustomLocatorHandler.java:113)
	at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
	at org.openqa.selenium.grid.node.Node.execute(Node.java:240)
	at org.openqa.selenium.grid.web.CombinedHandler.execute(CombinedHandler.java:59)
	at org.openqa.selenium.grid.web.RoutableHttpClientFactory$1.execute(RoutableHttpClientFactory.java:72)
	at org.openqa.selenium.grid.web.ReverseProxyHandler.execute(ReverseProxyHandler.java:92)
	at org.openqa.selenium.grid.router.HandleSession.execute(HandleSession.java:121)
	at org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)
	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
	at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
	at org.openqa.selenium.grid.router.Router.execute(Router.java:91)
	at org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)
	at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
	at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
	at org.openqa.selenium.remote.http.Route$NestedRoute.handle(Route.java:270)
	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
	at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
	at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
	at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
	at org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)
	at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
	at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
	at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
	at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
	at org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.openqa.selenium.json.JsonException: Expected to read a START_MAP but instead have: END. Last 0 characters read: 
Build info: version: '4.1.3', revision: '7b1ebf28ef'
System info: host: '83ee97fd6958', ip: '172.18.0.4', os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.0-107-generic', java.version: '11.0.14.1'
Driver info: driver.version: unknown
	at org.openqa.selenium.json.JsonInput.expect(JsonInput.java:307)
	at org.openqa.selenium.json.JsonInput.beginObject(JsonInput.java:229)
	at org.openqa.selenium.json.MapCoercer.lambda$apply$1(MapCoercer.java:64)
	at org.openqa.selenium.json.JsonTypeCoercer.lambda$buildCoercer$6(JsonTypeCoercer.java:145)
	at org.openqa.selenium.json.JsonTypeCoercer.coerce(JsonTypeCoercer.java:127)
	at org.openqa.selenium.json.Json.toType(Json.java:71)
	at org.openqa.selenium.json.Json.toType(Json.java:55)
	... 35 more

mghantous avatar Apr 11 '22 16:04 mghantous

Sadly the stacktrace is not giving us enough information to reproduce it. Would it be possible to get some tests that reproduce this? It is completely ok if I need to run them a few times and for a while to get this happening.

diemol avatar Apr 11 '22 17:04 diemol

Hi @diemol, just wondering if you tried this scenario: Available nodes = 30 Thread count = 40 (or any count as long as thread count > available nodes) This is one of the scenarios where I can consistently replicate the issue. The first batch seems to run properly but as the list in the queue grows, the error count grow as well.

chinzzz avatar Apr 18 '22 03:04 chinzzz

@chinzzz can you share the commands you are using to start the Grid?

diemol avatar Apr 20 '22 07:04 diemol

Hi, I just want to report that we see the same issue with version 4.1.3. It fails on random places when running our suit of tests that takes ~1h to run but I have not been able to reproduce it with a simple example. Our tests run just fine on 3.141.59.

Is there any debugloggin or something like that we could enable in order to find the source of this issue?

martenbohlin avatar Apr 29 '22 22:04 martenbohlin

@martenbohlin it'd be great to get a way to reproduce the issue.

You can also share the full logs by enabling debug level: SE_OPTS=--log-level FINE

diemol avatar May 04 '22 13:05 diemol

Hi, we're suffering from the same problem in ovirt-system-tests. Sometimes it's a JsonException, sometimes a TimeoutException, sometimes the Expected to read a START_MAP... thingy.

I've tried 4.0.0, 4.1.2, 4.1.4 both in a grid setup and with standalone nodes - it doesn't seem to matter. What does seem to matter is that recently we've moved the selenium containers from a physical host to a more restricted VM (2 vcpus, 2048M RAM) and it started to fail much more often since then.

If that's any help, here are FINE logs from a standalone chrome container. The stack trace was:

Stacktrace:
java.lang.RuntimeException: Unable to execute request for an existing session: java.util.concurrent.TimeoutException
Build info: version: '4.0.0', revision: '3a21814679'
System info: host: 'aa9a96dc77c4', ip: '10.0.2.100', os.name: 'Linux', os.arch: 'amd64', os.version: '4.18.0-383.el8.x86_64', java.version: '11.0.11'
Driver info: driver.version: unknown
        at org.openqa.selenium.grid.router.HandleSession.execute(HandleSession.java:139)
        at org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.grid.router.Router.execute(Router.java:91)
        at org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)
        at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)
        at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
        at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
        at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
        at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
        at org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.util.concurrent.TimeoutException
        at java.base/java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1886)
        at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2021)
        at org.asynchttpclient.netty.NettyResponseFuture.get(NettyResponseFuture.java:206)
        at org.openqa.selenium.remote.http.netty.NettyHttpHandler.makeCall(NettyHttpHandler.java:66)
        at org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)
        at org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:56)
        at org.openqa.selenium.remote.http.netty.NettyHttpHandler.execute(NettyHttpHandler.java:51)
        at org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)
        at org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:56)
        at org.openqa.selenium.remote.http.netty.NettyClient.execute(NettyClient.java:119)
        at org.openqa.selenium.remote.tracing.TracedHttpClient.execute(TracedHttpClient.java:55)
        at org.openqa.selenium.grid.web.ReverseProxyHandler.execute(ReverseProxyHandler.java:92)
        at org.openqa.selenium.grid.node.ProtocolConvertingSession.execute(ProtocolConvertingSession.java:75)
        at org.openqa.selenium.grid.node.local.SessionSlot.execute(SessionSlot.java:123)
        at org.openqa.selenium.grid.node.local.LocalNode.executeWebDriverCommand(LocalNode.java:388)
        at org.openqa.selenium.grid.node.ForwardWebDriverCommand.execute(ForwardWebDriverCommand.java:35)
        at org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.tracing.SpanWrappedHttpHandler.execute(SpanWrappedHttpHandler.java:86)
        at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.grid.node.Node.execute(Node.java:240)
        at org.openqa.selenium.grid.web.CombinedHandler.execute(CombinedHandler.java:59)
        at org.openqa.selenium.grid.web.RoutableHttpClientFactory$1.execute(RoutableHttpClientFactory.java:72)
        at org.openqa.selenium.grid.web.ReverseProxyHandler.execute(ReverseProxyHandler.java:92)
        at org.openqa.selenium.grid.router.HandleSession.execute(HandleSession.java:110)
        ... 22 more

I could describe a solution which would allow you to reproduce this, but it's definitely not trivial (i.e. you'd need a dedicated, physical host capable of doing virtualization).

Any help will be greatly appreciated, thanks!

tinez avatar May 06 '22 08:05 tinez

I get this same issue fairly frequently in a variety of locations in my code. To make it more simple - this is not a grid issue. I use the standalone firefox 4.1.2. I've seen the behavior since 4.0 and it has not improved. It's not something that is easily reproducible but it happens in our continuous test at least once a day. My stacks look like this -

Build info: version: '4.1.2', revision: '9a5a329c5a'
System info: host: 'ebcprh01886620-n.fyre.ibm.com', ip: '10.51.19.197', os.name: 'Linux', os.arch: 'amd64', os.version: '4.15.0-167-generic', java.version: '11.0.14.1'
Driver info: driver.version: RemoteWebDriver
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at org.openqa.selenium.remote.codec.w3c.W3CHttpResponseCodec.createException(W3CHttpResponseCodec.java:200)
at org.openqa.selenium.remote.codec.w3c.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:133)
at org.openqa.selenium.remote.codec.w3c.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:53)
at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:184)
at org.openqa.selenium.remote.TracedCommandExecutor.execute(TracedCommandExecutor.java:51)
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:558)
at org.openqa.selenium.remote.ElementLocation$ElementFinder$2.findElement(ElementLocation.java:162)
at org.openqa.selenium.remote.ElementLocation.findElement(ElementLocation.java:60)
at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:382)
at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:374)
at org.openqa.selenium.support.ui.ExpectedConditions$7.apply(ExpectedConditions.java:197)
at org.openqa.selenium.support.ui.ExpectedConditions$7.apply(ExpectedConditions.java:193)
at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:208)

amyreit avatar May 06 '22 13:05 amyreit

@tinez the log does not show the exception related to this issue. Do you have a log that shows that and ideally a way to reproduce it?

@amyreit is there a way you can share the complete logs?

Folks, if you just share the last piece of the log there is absolutely no way we can understand how the Grid got to that point.

diemol avatar May 07 '22 10:05 diemol

I can also see this error in 4.1.4 happening always in each build, sadly I can't give a reproducer. The first suspicious log in the grid is:

07:09:50.435 WARN [SeleniumSpanExporter$1.lambda$export$0] - {"traceId": "9f4b393d2e704ff3437e305a5551cc65","eventTime": 1651993790435123807,"eventName": "exception","attributes": {"exception.message": "Unable to execute request for an existing session: Unable to parse: ","exc
eption.stacktrace": "org.openqa.selenium.json.JsonException: Unable to parse: \n\tat org.openqa.selenium.json.Json.toType(Json.java:57)\n\tat org.openqa.selenium.json.Json.toType(Json.java:50)\n\tat org.openqa.selenium.grid.node.CustomLocatorHandler.execute(CustomLocatorHandle
r.java:113)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.grid.node.Node.execute(Node.java:240)\n\tat org.openqa.selenium.grid.web.CombinedHandler.ex
ecute(CombinedHandler.java:59)\n\tat org.openqa.selenium.grid.web.RoutableHttpClientFactory$1.execute(RoutableHttpClientFactory.java:72)\n\tat org.openqa.selenium.grid.web.ReverseProxyHandler.execute(ReverseProxyHandler.java:92)\n\tat org.openqa.selenium.grid.router.HandleSess
ion.execute(HandleSession.java:122)\n\tat org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.
openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.grid.router.Router.execute(Router.java:91)\n\tat org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)\n\tat org.openqa.selen
ium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)\n\tat org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)\n\tat org.openqa.seleni
um.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(
SeleniumHandler.java:44)\n\tat java.base\u002fjava.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base\u002fjava.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
Executor.java:1128)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base\u002fjava.lang.Thread.run(Thread.java:829)\nCaused by: org.openqa.selenium.json.JsonException: Expected to read a START_MAP but instead have
: END. Last 0 characters read: \nBuild info: version: '4.1.4', revision: '535d840ee2'\nSystem info: host: '72e53d629620', ip: '172.19.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1005-azure', java.version: '11.0.15'\nDriver info: driver.version: unknown\n\tat
org.openqa.selenium.json.JsonInput.expect(JsonInput.java:307)\n\tat org.openqa.selenium.json.JsonInput.beginObject(JsonInput.java:229)\n\tat org.openqa.selenium.json.MapCoercer.lambda$apply$1(MapCoercer.java:64)\n\tat org.openqa.selenium.json.JsonTypeCoercer.lambda$buildCoerce
r$6(JsonTypeCoercer.java:145)\n\tat org.openqa.selenium.json.JsonTypeCoercer.coerce(JsonTypeCoercer.java:127)\n\tat org.openqa.selenium.json.Json.toType(Json.java:71)\n\tat org.openqa.selenium.json.Json.toType(Json.java:55)\n\t... 31 more\n","exception.type": "org.openqa.selen
ium.json.JsonException","http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.router.HandleSession","http.host": "development-selenium.travelc.internal:4444","http.method": "POST","http.request_content_length": "59","http.scheme": "HTTP","http.target": "\u002fsessio
n\u002f70f5920659faf12b7d09bcfd8d700bb6\u002felement","http.user_agent": "selenium\u002f4.1.3 (java unix)","session.id": "70f5920659faf12b7d09bcfd8d700bb6"}}

Right before this one I can see these logs:


07:09:49.435 WARN [SeleniumSpanExporter$1.lambda$export$0] - {"traceId": "9f1d03c7412156479a30552651341df0","eventTime": 1651993789434909509,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.remote.http.Ro
ute$PredicatedRoute","http.host": "development-selenium.travelc.internal:4444","http.method": "POST","http.request_content_length": "48780","http.scheme": "HTTP","http.status_code": 404,"http.target": "\u002fsession\u002f347e8607f391badc6c818044aed6b6a1\u002fexecute\u002fsync"
,"http.user_agent": "selenium\u002f4.1.3 (java unix)"}}

07:09:49.448 WARN [SeleniumSpanExporter$1.lambda$export$0] - {"traceId": "45ab6ed4b71210ef653e543e3b730be8","eventTime": 1651993789447925704,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.remote.http.Ro
ute$PredicatedRoute","http.host": "development-selenium.travelc.internal:4444","http.method": "GET","http.scheme": "HTTP","http.status_code": 404,"http.target": "\u002fsession\u002f347e8607f391badc6c818044aed6b6a1\u002felement\u002f3ee249c2-9a03-47d7-8eef-a97d1227d972\u002ftex
t","http.user_agent": "selenium\u002f4.1.3 (java unix)"}}

07:09:49.457 WARN [SeleniumSpanExporter$1.lambda$export$0] - {"traceId": "2c6e4c285ea6c97c4d42199adb8645bb","eventTime": 1651993789456438910,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.remote.http.Ro
ute$PredicatedRoute","http.host": "development-selenium.travelc.internal:4444","http.method": "GET","http.scheme": "HTTP","http.status_code": 404,"http.target": "\u002fsession\u002f347e8607f391badc6c818044aed6b6a1\u002felement\u002f3ee249c2-9a03-47d7-8eef-a97d1227d972\u002fnam
e","http.user_agent": "selenium\u002f4.1.3 (java unix)"}}

Logs from the client side are not really relevant, I can see that any command is affected... createSession, findElements...

cocorossello avatar May 08 '22 07:05 cocorossello

@diemol Please see my repo for the updated stacktrace and standalone chrome logs.

As for the reproducibility - I had a ~85% success rate on my own server on Friday and today I got nothing... but we still see these issues in our CI infrastructure.

tinez avatar May 09 '22 13:05 tinez

It happens in many different places but here is one - trying to verify that icon is correct on the screen -

org.openqa.selenium.WebDriverException: Expected to read a START_MAP but instead have: END. Last 0 characters read: 
Build info: version: '4.1.2', revision: '9a5a329c5a'
System info: host: '2a897ed656a0', ip: '172.17.0.5', os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.0-99-generic', java.version: '11.0.13'
Driver info: driver.version: unknown
Build info: version: '4.1.2', revision: '9a5a329c5a'
System info: host: 'ebcprh01886620-n.fyre.ibm.com', ip: '10.51.19.197', os.name: 'Linux', os.arch: 'amd64', os.version: '4.15.0-167-generic', java.version: '11.0.14.1'
Driver info: org.openqa.selenium.remote.RemoteWebDriver
Command: [d9fb091a-a96b-447f-ae30-9bc9fc2948d1, findElement {using=id, value=collectionView-applications-card-app3(clusterA)newCard-cardIcon}]
Capabilities {acceptInsecureCerts: true, browserName: firefox, browserVersion: 96.0.3, javascriptEnabled: true, moz:accessibilityChecks: false, moz:buildID: 20220126154723, moz:debuggerAddress: localhost:49647, moz:firefoxOptions: {}, moz:geckodriverVersion: 0.30.0, moz:headless: false, moz:processID: 158, moz:profile: /tmp/rust_mozprofileNEAdzb, moz:shutdownTimeout: 60000, moz:useNonSpecCompliantPointerOrigin: false, moz:webdriverClick: true, pageLoadStrategy: normal, platform: LINUX, platformName: LINUX, platformVersion: 5.4.0-99-generic, proxy: Proxy(), se:cdp: ws://172.17.0.5:4444/sessio..., se:cdpVersion: 85.0, se:vnc: ws://172.17.0.5:4444/sessio..., se:vncEnabled: true, se:vncLocalAddress: ws://172.17.0.5:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify}
Session ID: d9fb091a-a96b-447f-ae30-9bc9fc2948d1
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at org.openqa.selenium.remote.codec.w3c.W3CHttpResponseCodec.createException(W3CHttpResponseCodec.java:200)
	at org.openqa.selenium.remote.codec.w3c.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:133)
	at org.openqa.selenium.remote.codec.w3c.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:53)
	at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:184)
	at org.openqa.selenium.remote.TracedCommandExecutor.execute(TracedCommandExecutor.java:51)
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:558)
	at org.openqa.selenium.remote.ElementLocation$ElementFinder$2.findElement(ElementLocation.java:162)
	at org.openqa.selenium.remote.ElementLocation.findElement(ElementLocation.java:60)
	at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:382)
	at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:374)
	at org.openqa.selenium.support.ui.ExpectedConditions$7.apply(ExpectedConditions.java:197)
	at org.openqa.selenium.support.ui.ExpectedConditions$7.apply(ExpectedConditions.java:193)
	at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:208)
	at com.ibm.ws.ui.fat.selenium.tests.AbstractUITest.waitForVisibilityOfElementLocatedBy(AbstractUITest.java:2134)
	at com.ibm.ws.ui.fat.selenium.tests.AbstractUITest.waitAndVerifyAttributeValueBy(AbstractUITest.java:2852)
	at com.ibm.ws.ui.fat.selenium.tests.AbstractUITest.waitAndVerifyAttributeValueBy(AbstractUITest.java:2806)
	at com.ibm.ws.ui.fat.selenium.tests.AbstractUITest.waitAndVerifyAttributeValueBy(AbstractUITest.java:2781)
	at com.ibm.ws.ui.fat.selenium.tests.AbstractUITest.waitAndVerifyAttributeValueById(AbstractUITest.java:2690)
	at com.ibm.ws.ui.fat.selenium.tests.AbstractUITest.waitAndVerifyAttributeValueById(AbstractUITest.java:2652)
	at com.ibm.ws.ui.fat.selenium.tests.AbstractUITest.verifySVG(AbstractUITest.java:3117)
	at com.ibm.ws.ui.fat.selenium.tests.explore.ExploreAbstractUITest.confirmAppCardInfo(ExploreAbstractUITest.java:1564)
	at com.ibm.ws.ui.tool.explore.fat.gui.nd.rw.tests.PartiallyRunningTest.appCardStarted(PartiallyRunningTest.java:239)
	at com.ibm.ws.ui.tool.explore.fat.gui.nd.rw.tests.PartiallyRunningTest.confirmAppCardData(PartiallyRunningTest.java:177)
	at com.ibm.ws.ui.tool.explore.fat.gui.nd.rw.tests.PartiallyRunningTest.appsResourceFilterTest(PartiallyRunningTest.java:121)
	at com.ibm.ws.ui.tool.explore.fat.gui.nd.rw.tests.PartiallyRunningTest.appsViewTest(PartiallyRunningTest.java:77)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:18)
	at componenttest.custom.junit.runner.FATRunner$1.evaluate(FATRunner.java:197)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:18)
	at componenttest.custom.junit.runner.FATRunner$2.evaluate(FATRunner.java:318)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
	at componenttest.custom.junit.runner.FATRunner.run(FATRunner.java:171)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:24)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:18)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
	at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:520)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1060)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:911)
[05/05/2022 15:01:44:969 PDT] 001 FATRunner                      evaluate                       I exiting com.ibm.ws.ui.tool.explore.fat.gui.nd.rw.tests.PartiallyRunningTest.appsViewTest

amyreit avatar May 09 '22 23:05 amyreit

Maybe this helps, we run version 4.1.4 and have the start-map issue as well in the selenium-hub. I cannot reproduce

06:39:12.031 WARN [SpanWrappedHttpHandler.execute] - Unable to execute request: Expected to read a START_MAP but instead have: END. Last 0 characters read:

Tue, May 10 2022 8:39:12 am | Build info: version: '4.1.4', revision: '535d840ee2'
Tue, May 10 2022 8:39:12 am | System info: host: 'selenium-hub-99d55b447-v8zt4', ip: '10.42.4.114', os.name: 'Linux', os.arch: 'amd64', os.version: '4.1.12-124.57.1.el7uek.x86_64', java.version: '11.0.15'
Tue, May 10 2022 8:39:12 am | Driver info: driver.version: unknown
Tue, May 10 2022 8:39:12 am | org.openqa.selenium.json.JsonException: Expected to read a START_MAP but instead have: END. Last 0 characters read:
Tue, May 10 2022 8:39:12 am | Build info: version: '4.1.4', revision: '535d840ee2'
Tue, May 10 2022 8:39:12 am | System info: host: 'selenium-hub-99d55b447-v8zt4', ip: '10.42.4.114', os.name: 'Linux', os.arch: 'amd64', os.version: '4.1.12-124.57.1.el7uek.x86_64', java.version: '11.0.15'
Tue, May 10 2022 8:39:12 am | Driver info: driver.version: unknown
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.json.JsonInput.expect(JsonInput.java:307)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.json.JsonInput.beginObject(JsonInput.java:229)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.NewSessionPayload.getOss(NewSessionPayload.java:304)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.NewSessionPayload.<init>(NewSessionPayload.java:97)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.NewSessionPayload.create(NewSessionPayload.java:76)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.grid.data.SessionRequest.<init>(SessionRequest.java:65)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.grid.sessionqueue.NewSessionQueue.lambda$new$0(NewSessionQueue.java:66)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.http.Route$TemplatizedRoute.handle(Route.java:192)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.grid.sessionqueue.NewSessionQueue.execute(NewSessionQueue.java:120)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.tracing.SpanWrappedHttpHandler.execute(SpanWrappedHttpHandler.java:86)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.grid.router.Router.execute(Router.java:91)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
Tue, May 10 2022 8:39:12 am | at org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)
Tue, May 10 2022 8:39:12 am | at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
Tue, May 10 2022 8:39:12 am | at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
Tue, May 10 2022 8:39:12 am | at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
Tue, May 10 2022 8:39:12 am | at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
Tue, May 10 2022 8:39:12 am | at java.base/java.lang.Thread.run(Thread.java:829)
Tue, May 10 2022 8:39:12 am | 06:39:12.031 WARN [SeleniumSpanExporter$1.lambda$export$0] - {"traceId": "8a20faa1ef02d639f48c423710757d73","eventTime": 1652164752031002739,"eventName": "exception","attributes": {"exception.message": "Unable to execute request: Expected to read a START_MAP but instead have: END. Last 0 characters read: \nBuild info: version: '4.1.4', revision: '535d840ee2'\nSystem info: host: 'selenium-hub-99d55b447-v8zt4', ip: '10.42.4.114', os.name: 'Linux', os.arch: 'amd64', os.version: '4.1.12-124.57.1.el7uek.x86_64', java.version: '11.0.15'\nDriver info: driver.version: unknown","exception.stacktrace": "org.openqa.selenium.json.JsonException: Expected to read a START_MAP but instead have: END. Last 0 characters read: \nBuild info: version: '4.1.4', revision: '535d840ee2'\nSystem info: host: 'selenium-hub-99d55b447-v8zt4', ip: '10.42.4.114', os.name: 'Linux', os.arch: 'amd64', os.version: '4.1.12-124.57.1.el7uek.x86_64', java.version: '11.0.15'\nDriver info: driver.version: unknown\n\tat org.openqa.selenium.json.JsonInput.expect(JsonInput.java:307)\n\tat org.openqa.selenium.json.JsonInput.beginObject(JsonInput.java:229)\n\tat org.openqa.selenium.remote.NewSessionPayload.getOss(NewSessionPayload.java:304)\n\tat org.openqa.selenium.remote.NewSessionPayload.\u003cinit>(NewSessionPayload.java:97)\n\tat org.openqa.selenium.remote.NewSessionPayload.create(NewSessionPayload.java:76)\n\tat org.openqa.selenium.grid.data.SessionRequest.\u003cinit>(SessionRequest.java:65)\n\tat org.openqa.selenium.grid.sessionqueue.NewSessionQueue.lambda$new$0(NewSessionQueue.java:66)\n\tat org.openqa.selenium.remote.http.Route$TemplatizedRoute.handle(Route.java:192)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.grid.sessionqueue.NewSessionQueue.execute(NewSessionQueue.java:120)\n\tat org.openqa.selenium.remote.tracing.SpanWrappe
  | dHttpHandler.execute(SpanWrappedHttpHandler.java:86)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.grid.router.Router.execute(Router.java:91)\n\tat org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)\n\tat org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)\n\tat java.base\u002fjava.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base\u002fjava.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base\u002fjava.lang.Thread.run(Thread.java:829)\n","exception.type": "org.openqa.selenium.json.JsonException","http.client_ip": "21.136.8.27","http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "selenium.bamf.intern","http.method": "POST","http.requ
  | est_content_length": "835","http.scheme": "HTTP","http.target": "\u002fsession","http.user_agent": "selenium\u002f3.141.5 (java unix)"}}
Tue, May 10 2022 8:39:12 am |  
Tue, May 10 2022 8:47:17 am | Trapped SIGTERM/SIGINT/x so shutting down supervisord...
Tue, May 10 2022 8:47:18 am | 2022-05-10 06:47:18,589 WARN received SIGTERM indicating exit request
Tue, May 10 2022 8:47:18 am | 2022-05-10 06:47:18,589 INFO waiting for selenium-grid-hub to die
Tue, May 10 2022 8:47:19 am | 2022-05-10 06:47:19,592 INFO stopped: selenium-grid-hub (terminated by SIGTERM)
Tue, May 10 2022 8:47:19 am | Shutdown complete

mschwehl avatar May 10 '22 07:05 mschwehl

This exception is thrown when the data that needs to be processed does not come in a properly formed JSON object. Which could be because of sync issues in the test, or the capabilities are incorrect, many different reasons. In short, the exception is correct.

From the stacktraces shared, I see it happens:

  • At org.openqa.selenium.grid.node.CustomLocatorHandler.execute(CustomLocatorHandler.java:113) because the contents of the request are empty.
  • Mostly when finding elements
  • One case in org.openqa.selenium.remote.NewSessionPayload.getOss(NewSessionPayload.java:304).

To continue the research, could you please share the test case where the exception happens? Even if it only happens every now and then.

diemol avatar May 10 '22 14:05 diemol

@diemol I also am having this same issue. The only way I can get my unresponsive grid back is to delete the router pod. Which is strange b/c the healthcheck should detect the unresponsive system and delete the pod.

kmcrawford avatar May 13 '22 16:05 kmcrawford

@kmcrawford can you share the stacktrace, please?

diemol avatar May 13 '22 19:05 diemol

My 2 cents, currently I am working from home, My internet connection losses a lot of packets, this is just an issue for activities such as online gaming, video streaming is fine. Interestingly, looks like selenium grid is packages dropped sensitive. since I run test executions from my home, Selenium grid complains a lot about this, however from my company offices everything works fine.

My recommendation, check from where you are executing your tests, try to avoid ssh proxies, the executing program should be in the same data center ideally

As how to replicate, I think it should be possible simulating this problem: https://stackoverflow.com/questions/614795/simulate-delayed-and-dropped-packets-on-linux

osdamv avatar May 19 '22 18:05 osdamv

We are experience the same problem when running tests in multiple parallel threads. Were able to reproduce with 4.0.0 and with the latest version - 4.1.4

We attempted to fix problem ourselves, below are results of our research (we used 4.0.0):

  1. Removed single threaded executor: https://github.com/SeleniumHQ/selenium/blob/trunk/java/src/org/openqa/selenium/netty/server/RequestConverter.java#L63 (I found that this is bottleneck in case when Netty configured to have multiple worker threads)
  2. Use FileBackedOutputStream instead of PipedInputStream/PipedOutputStream (not sure that this is affected on target result).
  3. Call ctx.fireChannelRead only after receive LastHttpContent instance.

These changes reduced START_MAP errors to less than 20 out of 1700 tests from 100-200 with out of the box implementation.

4.0.0 based patch is attached at https://gist.github.com/KupchinskiyVasiliy/af89e0fdf77fe06040708a3a3034bbf5

KupchinskiyVasiliy avatar May 20 '22 12:05 KupchinskiyVasiliy

Just tested with latest version 4.2.2-20220609 and I still can see the same error.

In client side:

org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Response code 500. Message: Expected to read a START_MAP but instead have: END. Last 0 characters read: 
Build info: version: '4.2.2', revision: '683ccb65d6'
System info: host: '2a3b30186f59', ip: '172.19.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1010-azure', java.version: '11.0.15'
Driver info: driver.version: unknown
Build info: version: '4.2.2', revision: '683ccb65d6'
System info: host: '515f0eef1232', ip: '172.20.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-1005-azure', java.version: '18.0.1'
Driver info: com.tr2.test.endtoend.selenium.base.IgnoreFirstConnectExceptionRemoteWebDriver
Command: [null, newSession {capabilities=[Capabilities {browserName: chrome, goog:chromeOptions: {args: [--window-size=1280,1280, --user-agent='Mozilla/5.0 (..., --disable-translate, --disable-extensions, --disable-background-networ..., --safebrowsing-disable-auto..., --disable-sync, --metrics-recording-only, --disable-default-apps, autoplay-policy=no-user-ges..., --disable-notifications, --disable-permissions-api, --ignore-certificate-errors, --force-prefers-reduced-motion, --headless, --headless, --use-gl=swiftshader, --disable-background-networ..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --blink-settings=imagesEnab...], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {intl.accept_languages: en-US,en,es}}, goog:loggingPrefs: org.openqa.selenium.logging...}], desiredCapabilities=Capabilities {browserName: chrome, goog:chromeOptions: {args: [--window-size=1280,1280, --user-agent='Mozilla/5.0 (..., --disable-translate, --disable-extensions, --disable-background-networ..., --safebrowsing-disable-auto..., --disable-sync, --metrics-recording-only, --disable-default-apps, autoplay-policy=no-user-ges..., --disable-notifications, --disable-permissions-api, --ignore-certificate-errors, --force-prefers-reduced-motion, --headless, --headless, --use-gl=swiftshader, --disable-background-networ..., --disable-background-timer-..., --disable-backgrounding-occ..., --disable-breakpad, --disable-client-side-phish..., --disable-component-extensi..., --disable-default-apps, --disable-features=TranslateUI, --disable-hang-monitor, --disable-ipc-flooding-prot..., --disable-popup-blocking, --disable-prompt-on-repost, --disable-renderer-backgrou..., --disable-sync, --force-color-profile=srgb, --metrics-recording-only, --no-first-run, --password-store=basic, --use-mock-keychain, --hide-scrollbars, --mute-audio, --blink-settings=imagesEnab...], excludeSwitches: [enable-automation, load-extension], extensions: [], prefs: {intl.accept_languages: en-US,en,es}}, goog:loggingPrefs: org.openqa.selenium.logging...}}]
Capabilities {}
	at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:144)
	at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:102)
	at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:67)
	at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:156)
	at org.openqa.selenium.remote.TracedCommandExecutor.execute(TracedCommandExecutor.java:51)
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:588)
	at com.tr2.test.endtoend.selenium.base.IgnoreFirstConnectExceptionRemoteWebDriver.execute(IgnoreFirstConnectExceptionRemoteWebDriver.java:28)
  	at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:283)

In the docker container side:

15:41:06.473 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "c44b03b8398aa79eafc38fff9a4e8a9d","eventTime": 1655134866473410678,"eventName": "exception","attributes": {"exception.message": "Unable to execute request for an existing session: org.asynchttpclient.exception.RemotelyClosedException: Remotely closed","exception.stacktrace": "java.io.UncheckedIOException: org.asynchttpclient.exception.RemotelyClosedException: Remotely closed\n\tat org.openqa.selenium.remote.http.netty.NettyHttpHandler.makeCall(NettyHttpHandler.java:73)\n\tat org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)\n\tat org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:56)\n\tat org.openqa.selenium.remote.http.netty.NettyHttpHandler.execute(NettyHttpHandler.java:49)\n\tat org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)\n\tat org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:56)\n\tat org.openqa.selenium.remote.http.netty.NettyClient.execute(NettyClient.java:98)\n\tat org.openqa.selenium.remote.tracing.TracedHttpClient.execute(TracedHttpClient.java:55)\n\tat org.openqa.selenium.grid.web.ReverseProxyHandler.execute(ReverseProxyHandler.java:92)\n\tat org.openqa.selenium.grid.node.ProtocolConvertingSession.execute(ProtocolConvertingSession.java:75)\n\tat org.openqa.selenium.grid.node.local.SessionSlot.execute(SessionSlot.java:125)\n\tat org.openqa.selenium.grid.node.local.LocalNode.executeWebDriverCommand(LocalNode.java:447)\n\tat org.openqa.selenium.grid.node.ForwardWebDriverCommand.execute(ForwardWebDriverCommand.java:35)\n\tat org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.tracing.SpanWrappedHttpHandler.execute(SpanWrappedHttpHandler.java:86)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.grid.node.Node.execute(Node.java:240)\n\tat org.openqa.selenium.grid.web.CombinedHandler.execute(CombinedHandler.java:59)\n\tat org.openqa.selenium.grid.web.RoutableHttpClientFactory$1.execute(RoutableHttpClientFactory.java:72)\n\tat org.openqa.selenium.grid.web.ReverseProxyHandler.execute(ReverseProxyHandler.java:92)\n\tat org.openqa.selenium.grid.router.HandleSession.execute(HandleSession.java:122)\n\tat org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.grid.router.Router.execute(Router.java:91)\n\tat org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$NestedRoute.handle(Route.java:270)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)\n\tat org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)\n\tat java.base\u002fjava.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base\u002fjava.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base\u002fjava.lang.Thread.run(Thread.java:829)\nCaused by: org.asynchttpclient.exception.RemotelyClosedException: Remotely closed\n\tat org.asynchttpclient.exception.RemotelyClosedException.INSTANCE(Unknown Source)\n","exception.type": "java.io.UncheckedIOException","http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.router.HandleSession","http.host": "selenium-spot.travelc.internal:4444","http.method": "POST","http.request_content_length": "68","http.scheme": "HTTP","http.target": "\u002fsession\u002fd8778d3279f1c53db071e5c39f86736e\u002fexecute\u002fsync","http.user_agent": "selenium\u002f4.2.2 (java unix)","session.id": "d8778d3279f1c53db071e5c39f86736e"}}

15:41:06.482 INFO [LocalSessionMap.lambda$new$0] - Deleted session from local Session Map, Id: d8778d3279f1c53db071e5c39f86736e
15:41:06.483 INFO [GridModel.release] - Releasing slot for session id d8778d3279f1c53db071e5c39f86736e
15:41:06.483 INFO [SessionSlot.stop] - Stopping session d8778d3279f1c53db071e5c39f86736e
15:41:06.586 WARN [SpanWrappedHttpHandler.execute] - Unable to execute request: org.asynchttpclient.exception.RemotelyClosedException: Remotely closed
java.io.UncheckedIOException: org.asynchttpclient.exception.RemotelyClosedException: Remotely closed
        at org.openqa.selenium.remote.http.netty.NettyHttpHandler.makeCall(NettyHttpHandler.java:73)
        at org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)
        at org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:56)
        at org.openqa.selenium.remote.http.netty.NettyHttpHandler.execute(NettyHttpHandler.java:49)
        at org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)
        at org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:56)
        at org.openqa.selenium.remote.http.netty.NettyClient.execute(NettyClient.java:98)
        at org.openqa.selenium.remote.tracing.TracedHttpClient.execute(TracedHttpClient.java:55)
        at org.openqa.selenium.grid.web.ReverseProxyHandler.execute(ReverseProxyHandler.java:92)
        at org.openqa.selenium.grid.node.ProtocolConvertingSession.execute(ProtocolConvertingSession.java:75)
        at org.openqa.selenium.grid.node.local.SessionSlot.execute(SessionSlot.java:125)
        at org.openqa.selenium.grid.node.local.LocalNode.executeWebDriverCommand(LocalNode.java:447)
        at org.openqa.selenium.grid.node.ForwardWebDriverCommand.execute(ForwardWebDriverCommand.java:35)
        at org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.tracing.SpanWrappedHttpHandler.execute(SpanWrappedHttpHandler.java:86)
        at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.grid.node.Node.execute(Node.java:240)
        at org.openqa.selenium.grid.web.CombinedHandler.execute(CombinedHandler.java:59)
        at org.openqa.selenium.grid.web.RoutableHttpClientFactory$1.execute(RoutableHttpClientFactory.java:72)
        at org.openqa.selenium.grid.web.ReverseProxyHandler.execute(ReverseProxyHandler.java:92)
        at org.openqa.selenium.grid.router.HandleSession.execute(HandleSession.java:122)
        at org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.grid.router.Router.execute(Router.java:91)
        at org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)
        at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.http.Route$NestedRoute.handle(Route.java:270)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)
        at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
        at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
        at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
        at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
        at org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.asynchttpclient.exception.RemotelyClosedException: Remotely closed
        at org.asynchttpclient.exception.RemotelyClosedException.INSTANCE(Unknown Source)
15:41:06.586 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to execute request: org.asynchttpclient.exception.RemotelyClosedException: Remotely closed

cocorossello avatar Jun 13 '22 15:06 cocorossello

We have the same issue, selenium 4.2.2.

Could this be concurrency related maybe due to timeouts and waits? Although being discouraged we are using both, implicit and explicit waits. Additionally we are initializing firefox driver with this capabilities:

FirefoxOptions opt = new FirefoxOptions();
Map<String, Integer> timeouts = new HashMap<>();
timeouts.put("implicit", 3 * 1000);
opt.setCapability("timeouts", timeouts);

awaterme avatar Jul 09 '22 05:07 awaterme

I've got the same error randomly on the same line, however. print(column.find_element(By.TAG_NAME,'span').text, end=',')

So by add a little delay under the above line (time.sleep(0.1)) seems to works for me.... I don't know if I'm just lucky, but I was able to finish the scrap.

Traceback (most recent call last):
  File "/usr/src/app/./scraper.py", line 37, in <module>
    print(column.find_element(By.TAG_NAME,'span').text, end=',')
  File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/remote/webelement.py", line 419, in find_element
    return self._execute(Command.FIND_CHILD_ELEMENT,
  File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/remote/webelement.py", line 396, in _execute
    return self._parent.execute(command, params)
  File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 435, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Expected to read a START_MAP but instead have: END. Last 0 characters read: 
Build info: version: '4.3.0', revision: 'a4995e2c09*'
System info: host: '0347ee78d194', ip: '172.17.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.0-104-generic', java.version: '11.0.15'
Driver info: driver.version: unknown
Stacktrace:
org.openqa.selenium.json.JsonException: Expected to read a START_MAP but instead have: END. Last 0 characters read: 
Build info: version: '4.3.0', revision: 'a4995e2c09*'
System info: host: '0347ee78d194', ip: '172.17.0.2', os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.0-104-generic', java.version: '11.0.15'
Driver info: driver.version: unknown
        at org.openqa.selenium.json.JsonInput.expect(JsonInput.java:307)
        at org.openqa.selenium.json.JsonInput.beginObject(JsonInput.java:229)
        at org.openqa.selenium.json.MapCoercer.lambda$apply$1(MapCoercer.java:64)
        at org.openqa.selenium.json.JsonTypeCoercer.lambda$buildCoercer$6(JsonTypeCoercer.java:145)
        at org.openqa.selenium.json.JsonTypeCoercer.coerce(JsonTypeCoercer.java:127)
        at org.openqa.selenium.json.Json.toType(Json.java:71)
        at org.openqa.selenium.json.Json.toType(Json.java:55)
        at org.openqa.selenium.json.Json.toType(Json.java:50)
        at org.openqa.selenium.grid.node.CustomLocatorHandler.execute(CustomLocatorHandler.java:113)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.grid.node.Node.execute(Node.java:240)
        at org.openqa.selenium.grid.web.CombinedHandler.execute(CombinedHandler.java:59)
        at org.openqa.selenium.grid.web.RoutableHttpClientFactory$1.execute(RoutableHttpClientFactory.java:72)
        at org.openqa.selenium.grid.web.ReverseProxyHandler.execute(ReverseProxyHandler.java:92)
        at org.openqa.selenium.grid.router.HandleSession.execute(HandleSession.java:122)
        at org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.grid.router.Router.execute(Router.java:91)
        at org.openqa.selenium.grid.web.EnsureSpecCompliantResponseHeaders.lambda$apply$0(EnsureSpecCompliantResponseHeaders.java:34)
        at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.http.Route$NestedRoute.handle(Route.java:270)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)
        at org.openqa.selenium.remote.http.Route.execute(Route.java:68)
        at org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)
        at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
        at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
        at org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)
        at org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)
        at org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

sl45sms avatar Jul 09 '22 14:07 sl45sms

Getting this with 4.4.0. We're running 12 tests concurrently against Selenium Grid (4.4.0-20220812). Two of the tests received this error.

Client error:

Caused by: org.openqa.selenium.json.JsonException: Unable to parse: 400 Bad Request
	at app//org.openqa.selenium.json.Json.toType(Json.java:57)
	at app//org.openqa.selenium.json.Json.toType(Json.java:50)
	at app//org.openqa.selenium.remote.codec.w3c.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:99)
	at app//org.openqa.selenium.remote.codec.w3c.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:53)
	at app//org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:184)
	at app//org.openqa.selenium.remote.TracedCommandExecutor.execute(TracedCommandExecutor.java:51)
	at app//org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:547)
	at app//org.openqa.selenium.remote.RemoteWebElement.execute(RemoteWebElement.java:257)
	at app//org.openqa.selenium.remote.RemoteWebElement.getRect(RemoteWebElement.java:349)
	at app//dk.b.w.selenium.core.WebCore.waitForElementToStopMoving(WebCore.java:477)
	at app//dk.b.w.selenium.core.WebCore.waitForElementToStopMoving(WebCore.java:467)
	at app//dk.b.w.selenium.core.WebCore.waitForElementToStopMoving(WebCore.java:462)
	at app//dk.b.w.selenium.r.rc.clickMainAddButton(rc.java:478)
	at app//dk.b.w.selenium.r.v.initAddItem(v.java:123)
	at app//dk.b.w.selenium.r.v.addAccount(v.java:49)
	at app//dk.b.w.selenium.r.v.addOwnAccount(v.java:33)
	at app//dk.b.w.selenium.e2e.AdvisoryDraftTest.advisorCannotManipulateWhenNoBankConsent(AdvisoryDraftTest.java:407)
	at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at [email protected]/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at [email protected]/java.lang.reflect.Method.invoke(Method.java:566)
	at app//org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
	at app//org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
	at app//org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
	at app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
	at app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
	at app//org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
	at app//org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
	at app//org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
	at app//org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
	at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
	... 34 more
Caused by: org.openqa.selenium.json.JsonException: Expected to read a START_MAP but instead have: NUMBER. Last 0 characters read: 
Build info: version: '4.4.0', revision: 'e5c75ed026a'
System info: host: 'jdk11-builder-2z9nn-5b6km', ip: '10.13.128.23', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-1160.71.1.el7.x86_64', java.version: '11.0.11'
Driver info: driver.version: RemoteWebDriver
	at app//org.openqa.selenium.json.JsonInput.expect(JsonInput.java:307)
	at app//org.openqa.selenium.json.JsonInput.beginObject(JsonInput.java:229)
	at app//org.openqa.selenium.json.MapCoercer.lambda$apply$1(MapCoercer.java:64)
	at app//org.openqa.selenium.json.JsonTypeCoercer.lambda$buildCoercer$6(JsonTypeCoercer.java:145)
	at app//org.openqa.selenium.json.JsonTypeCoercer.coerce(JsonTypeCoercer.java:127)
	at app//org.openqa.selenium.json.Json.toType(Json.java:71)
	at app//org.openqa.selenium.json.Json.toType(Json.java:55)
	... 69 more

I have the following error on a Selenium Chrome Node on Selenium Grid:

07:01:06.085 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "4da48901e33452daa28a9d3ac17243f0","eventTime": 1662361266082704089,"eventName": "exception","attributes": {"exception.message": "Unable to execute request: org.asynchttpclient.exception.RemotelyClosedException: Remotely closed","exception.stacktrace": "java.io.UncheckedIOException: org.asynchttpclient.exception.RemotelyClosedException: Remotely closed\n\tat org.openqa.selenium.remote.http.netty.NettyHttpHandler.makeCall(NettyHttpHandler.java:73)\n\tat org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)\n\tat org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:56)\n\tat org.openqa.selenium.remote.http.netty.NettyHttpHandler.execute(NettyHttpHandler.java:49)\n\tat org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)\n\tat org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:56)\n\tat org.openqa.selenium.remote.http.netty.NettyClient.execute(NettyClient.java:98)\n\tat org.openqa.selenium.remote.tracing.TracedHttpClient.execute(TracedHttpClient.java:55)\n\tat org.openqa.selenium.grid.web.ReverseProxyHandler.execute(ReverseProxyHandler.java:92)\n\tat org.openqa.selenium.grid.node.ProtocolConvertingSession.execute(ProtocolConvertingSession.java:75)\n\tat org.openqa.selenium.grid.node.local.SessionSlot.execute(SessionSlot.java:125)\n\tat org.openqa.selenium.grid.node.local.LocalNode.executeWebDriverCommand(LocalNode.java:448)\n\tat org.openqa.selenium.grid.node.ForwardWebDriverCommand.execute(ForwardWebDriverCommand.java:35)\n\tat org.openqa.selenium.remote.http.Route$PredicatedRoute.handle(Route.java:373)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.tracing.SpanWrappedHttpHandler.execute(SpanWrappedHttpHandler.java:86)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.grid.node.Node.execute(Node.java:240)\n\tat org.openqa.selenium.remote.http.Route$CombinedRoute.handle(Route.java:336)\n\tat org.openqa.selenium.remote.http.Route.execute(Route.java:68)\n\tat org.openqa.selenium.remote.AddWebDriverSpecHeaders.lambda$apply$0(AddWebDriverSpecHeaders.java:35)\n\tat org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.remote.ErrorFilter.lambda$apply$0(ErrorFilter.java:44)\n\tat org.openqa.selenium.remote.http.Filter$1.execute(Filter.java:64)\n\tat org.openqa.selenium.netty.server.SeleniumHandler.lambda$channelRead0$0(SeleniumHandler.java:44)\n\tat java.base\u002fjava.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base\u002fjava.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base\u002fjava.lang.Thread.run(Thread.java:829)\nCaused by: org.asynchttpclient.exception.RemotelyClosedException: Remotely closed\n\tat org.asynchttpclient.exception.RemotelyClosedException.INSTANCE(Unknown Source)\n","exception.type": "java.io.UncheckedIOException","http.client_ip": "10.197.103.126","http.flavor": 1,"http.handler_class": "org.openqa.selenium.remote.http.Route$PredicatedRoute","http.host": "selenium-hub-wea-staging.apps.c02u.paastools.bdunet.dk","http.method": "POST","http.request_content_length": "50","http.scheme": "HTTP","http.target": "\u002fsession\u002f673c880280f4448821cb7e2f6c01a74b\u002felement\u002f92cbbcde-5e60-4d5b-860a-e3c05cdbfb33\u002fclick","http.user_agent": "selenium\u002f4.4.0 (java unix)"}}

AndreasPetersen avatar Sep 05 '22 07:09 AndreasPetersen

Same problem here, with selenium 4.4.0, running as standalone server in combination with firefox. Log file with some context: selenium.log

awaterme avatar Sep 06 '22 06:09 awaterme

I was just able to reproduce this twice by having no nodes connected, passing in tests to make a queue, then adding a node after a few minutes. Failures with the error in the title occur, and the node that launches immediately gets a blank browser that just sits at the address "Data:," if in chrome.

In the wild, this occurs during a test run that hits a queue. There will be multiple browsers that hang, open to the address of "Data;." (if Chrome) and tests will fail with org.openqa.selenium.json.JsonException: Expected to read a START_MAP but instead have: END. Last 0 characters read:. The test will report failed, but the node will hang at the "Data:," address for about 10 minutes if running chrome. If firefox, same thing, just a blank browser. It seems each test coming out of the queue will hang like this, meaning less nodes are available as more come in and enter the hung state.

We previously ran Selenium 3 without this issue, however if a test was in the queue too long it would simply fail with "Failed all retries to get a browser session".

The setup is a fully distributed model (router, distributor with event bus, session map, session queue) on different VMs. The nodes are effectively ran as provided by docker-selenium, however each one has Chrome+Edge+Firefox. Autoscaling is used to scale the number of nodes. Nodes and each grid component are on v4.4.0.

I'm wondering if this is a case of:

  • no nodes available
  • test is sent to router
  • node request for test goes queue
  • because no nodes are available, eventually test times out and marks itself as failed
  • request remains in queue, eventually node becomes available
  • node request takes a browser, but does not receive any instruction as test has already failed

Curious if anyone shares a similar deployment or can reproduce the issue in the same way.

efranken avatar Sep 14 '22 21:09 efranken