Does not create multiple connections to back end gRPC service

[newmans99]

Description

I am working on building a gRPC Python container to provide specific functions to the Qlik Engine. The service works correctly and I get results back without problems. The challenge is that when I try to scale it out to large number of data sets (App Reload function calls) OR with large numbers of users (App Sheet/Visualization calls), although I have 10 load balanced containers running, only the first one that is attached to by the engine is used.

I am not sure if this is caused by the Python Connector, gRPC, or by the way that the Qlik Engine interacts with the gRPC Client. Can anyone let me know where I should start looking to help build out the scalability that is required for large scale applications?

[tobiaslindulf]

Thanks for the comment and input. We are investigating this and will update this issue when we have a solid response. What we can say for sure is that Qlik has not used/implemented any Client-side load balancing logic. In other words the only way of load balancing that could work is the server-side LB (proxy). More info will come after the investigations.

[nabeel-oz]

Hi, have there been any changes in handling parallel calls since this issue was opened?

I'm testing my Python SSE which is based on __main__.py from the Python column operations examples. When I have multiple SSE calls on the same sheet, they seem to be triggered one by one. I get spinning circles on the different visualizations, and then the circles go off one by one and finally all the visualizations are rendered. The SSE terminal shows the commands being executed sequentially as well.

But if I use another app that calls a different function from the same SSE at the same time, it runs in parallel and can give me results before the app with the more intensive function has completed rendering the visualizations.

If you want access to the SSE and sample apps to see what I mean I'm happy to share my project.

[rileymd88]

Any update on this issue?

[pamatulli81]

We also have encountered performance and stability problem related to te SSE integration due to the sequencial processing of requests, especially when multiple users are accessing the Qlik Sense App. Is there any alternative. Do we have any chance to improve this behaviour without setting up a load balancing mechanism with multiple instances?

[AnalyticsEarth]

@pamaxeed

The engine will send requests asynchronously, so should not serial bottle neck. You might have issues in the downstream code. What type of connector are you using?

Load balancing requires a level 7 load balancer as GRPC uses HTTP2.0. Something like NGINX can be used. I am working on a SSE performance white paper at the moment and will be out in the coming month with a view of how to scale. Would be interested to know your experience and your setup. Feel free to reach out at steven.pressland at qlik.com

[baffelli]

In the SSE examples, the server is started using a ThreadPoolExecutor. Because of python's GIL (Global Interpreter Lock), this means that the server is capable to serve multiple users concurrently but not in parallel.

With a ThreadPool, the SSE server can process an I/O bound task (i.e receive a new computation request) while the computation runs, but it cannot run multiple computations in parallel.

The SSE uses a gRPC server running on a thread pool (see here) in order to process the requests in separate threads. The problem with this approach is that as soon as one of these requests starts a CPU task (a computation), it will be blocking all other previously started CPU tasks, since python only supports one CPU-bound thread. Thus, instead of serving n users in parallel, this only causes a "race" for the CPU thread among the concurrent users.

A better approach would be to use a queue to regulate the order of execution of CPU-bound task and to process requests in a First-come-First-Serve basis.

Another solution is to initialize a ProcessPoolExector to run the CPU tasks in separate processes. However, in this case only as many users as the number of CPUs can be served in parallel.