aerospike-client-python icon indicating copy to clipboard operation
aerospike-client-python copied to clipboard

Memory consumption and number of threads created

Open germannocr opened this issue 5 years ago • 1 comments

I am currently part of a team that develops a Python Rest API and uses Aerospike as a caching feature. However, its initialization has significantly increased the memory consumption and the number of threads, without any processing having been done. Only on startup does the number of threads increase from 2 to 18. Does this make sense? Is there any configuration that allows me to define how many threads will be used? My startup and client connection is correct according to the official documentation.

germannocr avatar Jul 17 '20 13:07 germannocr

Hi @germannocr ,

The Python client uses a thread pool in order to send synchronous batch, scan, and query requests to multiple server nodes in parallel threads. The default thread count is 16, it can be configured upon client creation using the thread_pool_size config. You can actually set this to 0 if all node commands are sent sequentially. That means making sure all scans do not set the concurrent policy to true. Same for batch request's concurrent policy. Queries are always run concurrently so they cannot be used with a thread_pool_size of 0.

Thanks for posting and I hope this helps. :)

dwelch-spike avatar Jul 17 '20 19:07 dwelch-spike