Mock and Benchmark servers not able to handle multiplexed requests in parallel

[netofontes]

Mock and benchmark servers can't handle multiple requests in parallel when they are originated from the same connection. However, the problem doesn't happen when requests are originated from different connections.

This problem was discovered while performance testing our own client against a mock server with latency artificially introduced through a custom server handler.

This is causing all requests originated from the same client to be processed in sequential fashion which is impacting on our performance test results.

As both mock and benchmark servers supports HTTP/2, they should be able to handle multiplexed requests in parallel which was not observed when troubleshooting server handlers.

[jchambers]

I would describe this as a design decision rather than a bug. Can you please elaborate on your use case? For many cases—especially the benchmark case—the cost of dispatching requests to multiple threads and context-switching quickly approaches the cost of just processing the whole request.

[netofontes]

Yes, I can see that the overhead of handling context switch and dispatching jobs to threads would be significant and unnecessary if the mock server is not actually processing the received payload which is currently the case of the benchmark server where the server is pretty much doing a no-op.

However, our performance test use case needs to send invalid requests at random and we want to see how our client behaves (performance wise) upon invalid request responses under load condition. We also need to emulate random request timeouts under load condition too.

In order to implement the test case above, the mock server should no longer do a no-op on every request. With that change, it now makes sense to dispatch jobs to threads available in a thread pool so that subsequent requests from the same connection won't be impacted if previous requests are long running.

As a good compromise, maybe we could have an option in the the server builder to enable the handling of multiplexed requests in parallel. That option could even take a new thread pool or just reuse the existing internal thread pool already present in the server.

[jchambers]

I'm sorry for the long delay on this issue (five years!). While I think your point is valid, I think it's taking the mock server pretty far beyond its intended purpose, which is supporting testing within Pushy. It sounds like you have some pretty specific needs, and those needs are not likely to be shared by many other users.

While I understand it may be disappointing, I do not intend to modify the mock server to support concurrent request processing in the foreseeable future.