microsoft
memory usage issue
I have a long-running multi-threaded KV program that provides get and set interfaces. The program is configured with a soft_limit memory value. When the total memory usage of the data exceeds this value, each thread initiates an eviction operation, calling the mi_free function to delete a small portion of KV pairs, and then continues to periodically check.
I started an external stress testing program that continuously calls the set interface, with each value being 64MB in size, and each time new memory is allocated. The soft_limit is configured to 10GB. I observed that eviction operations began as expected when the memory usage exceeded 10GB. The data capacity remained fluctuating below 10GB, but from the system monitoring tool top, I noticed that the program's RSS (Resident Set Size) kept increasing. Although the initial increase to 10GB was rapid, the RSS continued to slowly increase afterwards. Is this behavior expected? How can I keep the program's RSS at a stable level instead of gradually increasing?
My mimalloc version is v2.1.7, linux platform.
Hmm -- it should stay stable so not sure what is going on. Can you try with the latest dev3 and see if the issue persists -- this branch has a more fine-grained sharing of memory between threads and generally uses less memory than v2.
hi, thank you for your reply, I will try it! By the way, there is tag v3.0.1. Is this tag version a stable version? Could I use it in production environment?
Ah, v3.0.1 is alpha and there have been a few important bug fixes so it is better to use the latest dev3. I will make a fresh release soon again that will be "beta" (but I think it is now getting quite robust)