gatling
HTTP: ws ping from client
Feature description
I need to check whether the websocket connection is still alive, and as far as I know, this isn't possible right now in Gatling. Ideally, we would be able to send ping message frames from within scenarios and check for a corresponding pong response. I know the client already responds to pings sent from the server, but this is the other way around.
Use cases
This can be used to test whether or not the current open websockets are still alive.
Test methods
I've tested whether this functionality was already present in Gatling by unplugging my ethernet cable whilst an idle websocket was open. The test results didn't show any indication of a websocket failure/closure and so this can easily lead to incomplete test results. I have also consulted the Gatling documentation, but nothing regarding this subject could be found.
Tested solutions
I've gotten the ping working with the code below, but it's SUPER hacky and doesn't check for a pong response. So it's pretty awful and not workable.
val ws = session("gatling.http.webSocket").as[WsFsm]
val field = classOf[WsFsm].getDeclaredField("currentState") // required to gain insight in Gatling's internal websocket state
field.setAccessible(true)
val fieldWs = classOf[WsIdleState].getDeclaredField("webSocket")
fieldWs.setAccessible(true)
fieldWs.get(field.get(ws).asInstanceOf[WsIdleState]).asInstanceOf[WebSocket].sendFrame(new PingWebSocketFrame())
First, are you sure you're referring to actual Ping WebSocket frames, and not applicative Text frames, like what socket.io does?
FYI, it's not possible to send a Ping frame in Javascript in a web browser so indeed, that's not something we support either.
It makes sense to have the server side (the side that didn't initiate the connection) check that the client peer is still there. Clients can't be trusted to well behave.
We do though automatically reply to a Ping frame with a Pong one (like a browser would). And in the next release, we'll also be able to automatically reply to some Text frames with another, to support the new keep-alive mechanism in socket.io 4 that has switch keep-alive initiation from the client to the server. See #3929
First, are you sure you're referring to actual Ping WebSocket frames, and not applicative Text frames, like what socket.io does?
I'm referring to actual ping websocket frames.
FYI, it's not possible to send a Ping frame in Javascript in a web browser so indeed, that's not something we support either.
It makes sense to have the server side (the side that didn't initiate the connection) check that the client peer is still there. Clients can't be trusted to well behave.
We do though automatically reply to a Ping frame with a Pong one (like a browser would). And in the next release, we'll also be able to automatically reply to some Text frames with another, to support the new keep-alive mechanism in socket.io 4 that has switch keep-alive initiation from the client to the server. See #3929
I see, but what would be the current solution for detecting whether websockets crash on the serverside? As outlined in my initial issue, when I unplug my ethernet cable during a scenario with open websockets there is no indication whatsoever in the test results that the websocket connection dropped - even subsequent ws().sendText("") messages do not get KO'd. The same occurs when I abruptly terminate the host websocket server - which could be a real-life result of an intense load-test.
In summary, Gatling should be able to detect and report these events in the reports in order to properly evaluate load tests, and a client -> server ping seems like the most logical solution to detect these disruptions. Currently, these disruptions can only be detected by sending actual data, either using sendText or sendBytes.
Additional comments
Also, shouldn't Gatling mark the corresponding actions that sent the request as failed (KO)? Currently, all that happens on a crashed websocket is a call to WsConnectingState.gotoConnecting, but the initial sendText request gets an OK regardless of the crashed websocket state. This might also lead to misleading results.
I see, but what would be the current solution for detecting whether websockets crash on the serverside? As outlined in my initial issue, when I unplug my ethernet cable during a scenario with open websockets there is no indication whatsoever in the test results that the websocket connection dropped - even subsequent ws().sendText("") messages do not get KO'd. The same occurs when I abruptly terminate the host websocket server - which could be a real-life result of an intense load-test.
@vJoeyz this is something similar, like I described here? https://github.com/gatling/gatling/issues/4116#issuecomment-979044065, instead getting KO'd the WS reconnects and send request and marked it as OK.
@vJoeyz this is something similar, like I described here? #4116 (comment), instead getting KO'd the WS reconnects and send request and marked it as OK.
It's similar, but even when the reconnect never succeeds the test still doesn't get KO'd.
@rjaros87 @vJoeyz Please stick to the 1 ticket = 1 concern rule. This ticket's title is "ws ping from client". If you run into an issue, please open a dedicated ticket and provide a Short, Self Contained, Correct (Compilable), Example.
@rjaros87 @vJoeyz Please stick to the 1 ticket = 1 concern rule. This ticket's title is "ws ping from client". If you run into an issue, please open a dedicated ticket and provide a Short, Self Contained, Correct (Compilable), Example.
I will, though the original request/concern remains.
I need to check whether the websocket connection is still alive, and as far as I know, this isn't possible right now in Gatling.
I'm not sure we'll implement a feature that doesn't mirror something the actual applications do. How do your actual client applications deal with closed connections?
I need to check whether the websocket connection is still alive, and as far as I know, this isn't possible right now in Gatling.
I'm not sure we'll implement a feature that doesn't mirror something the actual applications do. How do your actual client applications deal with closed connections?
They use a reconnect mechanism, but afaik Gatling doesn't detect a disconnect/crash when no text or byte messages are being sent. This results in no reconnection at all until the next message is sent, which then (as a result of the disconnect) gets KO'd.
Gatling currently lazily reconnects when sending a message or a check, not eagerly. IMHO, that’s the correct behavior. Note: there was a bug on reconnect that was fixed in the latest release.
Gatling currently lazily reconnects when sending a message or a check, not eagerly. IMHO, that’s the correct behavior. Note: there was a bug on reconnect that was fixed in the latest release.
Would you be open to receiving a contribution containing outlined functionality (initiating WebSocket ping frames and listening for a response)?