xk6-websockets

xk6-websockets copied to clipboard

I want to model a scenario where I have a set of VUs issuing requests repeatedly over the same websockets. It seems like this could be accomplished with something like:

var socket = null;

export default async function() {
    if (__ITER === 0) {
        socket = new WebSocket(url, params);
        ... 
    }
    socket.send("my message");
    ...

However, the test hangs forever, unless the underlying websocket is closed. Is there a way to prevent this behavior; allowing a websocket to remain open between iterations?

Hi @DerekTBrown,

I'm sorry, but I think there's currently no way to share a WebSocket across different VUs (nor iterations), because they're run as different JS VMs (and even they might or might not run on the same machine).

For patterns like the one you shared, I think the setup function could be the way to go. But still, in this case it won't work because the setup will not end until all of it's of event loop events are finished, so in this case this will block forever or until the websocket is stopped.

That said, would you be able to share the reason why you're trying to follow that pattern? If you use case is common among other users, we might explore ways to add support for it.

Thanks! 🙇🏻‍♂️

That said, would you be able to share the reason why you're trying to follow that pattern? If you use case is common among other users, we might explore ways to add support for it.

I am trying to test the behavior of a service when it receives an influx of traffic over the same, re-used websocket connections from a handful of callers. Ideally, I would be able to tell K6 the number of VUs (in this case, corresponding to the number of Websocket clients), and iterations (corresponding to the number of messages sent over each websocket).

As you mentioned, because clients cannot be shared across iterations, this isn't possible. I believe the best I can do is to configure perVuIterations with a single iteration, and then manage the messages inside a for loop. This obviously isn't ideal, since the metrics are tied to the number of iterations.

I also think an analogous case exists for gRPC streaming; you may want to measure the performance of messages sent over a pre-existing stream, rather than including the time to create and destroy the stream.

I also think an analogous case exists for gRPC streaming; you may want to measure the performance of messages sent over a pre-existing stream, rather than including the time to create and destroy the stream.

Do you actually have multiple threads re-using the same socket on your real client? Or the purpose of re-using the client is just to "sanitize" the measurements?

I guess that, in such case, you may want to have a long running iteration on each VU, and define your own custom metrics, if none of the ones defined by the xk6-websockets extension by default is enough for you.

Look at this docs, it might help you as a source inspiration, to customize the metrics as you need.

I have a similar use case, and that's pretty much what I do now: just have each iteration last a while and use setInterval to send messages repeatedly, and a setTimeout that closes the socket. What I really want is to be able to let k6 control the rate at which messages are sent so that I can use the arrival rate executors. Otherwise I'd have to script all that logic myself.

Do you actually have multiple threads re-using the same socket on your real client? Or the purpose of re-using the client is just to "sanitize" the measurements?

We do have clients that use connection pooling, as well as long-lived connections (as HTTP/2 and gRPC tend to do). We want to simulate those behaviors as closely as possible in K6, both because of client/server behavior, but also because the network stack behaves differently with long-lived connections.

I guess that, in such case, you may want to have a long running iteration on each VU, and define your own custom metrics, if none of the ones defined by the xk6-websockets extension by default is enough for you.

Similar to @dyc3 , we ended up with a primitive re-implementation of a lot of the executor control logic inside a single iteration. Because of this, we miss out on a lot of the core features of K6 (the standard set of metrics/thresholds don't work, we can't use the executor tuning tools, etc).

Hey @DerekTBrown,

Similar to @dyc3 , we ended up with a primitive re-implementation of a lot of the executor control logic inside a single iteration. Because of this, we miss out on a lot of the core features of K6 (the standard set of metrics/thresholds don't work, we can't use the executor tuning tools, etc).

Could you please bring more details about all this, so we the team can uses to determine how much effort deserves this issue and how feasible would be to cover certain aspects of your needs? 🙇🏻

Thanks!