`imap::entry_set` leaks memory

[yemreinci]

Run the following reproducer and check memory usage after every pause using ps -o pid,user,rss,size,command ax | grep <programname>.

#include <iostream>
#include <string>

#include <hazelcast/client/hazelcast_client.h>

void pause(const char* msg) {
    std::cout << msg << std::flush;
    std::cin.get();
}

int
main()
{
    {
        auto hz = hazelcast::new_client().get();
        auto map = hz.get_map("test_map").get();

        // fill map with ~100MBs of data
        for (int i = 0; i < 1024 * 100; i++) {
            map->set(std::to_string(i), std::string(1024, 'a')).get();
        }

        pause("press [enter] to start reading.");

        for (int i = 0; i < 10; i++) {
            auto entries = map->entry_set<std::string, std::string>().get();

            pause("press [enter] for next iteration.");
        }
    }

    pause("check usage after client is destroyed.");
    
    return 0;
}

[yemreinci]

Adding malloc_trim(0) after entry_set fixes the increasing memory usage, which means there is no real leak, i.e. the memory is freed after entry_set. But somehow the malloc implementation keeps allocating new pages in every iteration instead of reusing the previously freed space. I believe this is more of an issue with Glibc's malloc implementation than with the library but I will investigate more to see there is anything we can do to improve the situation.

Also, I observed that setting export GLIBC_TUNABLES=glibc.malloc.arena_max=1 seems to solve the increasing memory usage.

[akeles85]

I don't see any memory leakage with version 5.2. As the issue reporter told it might be releated with glibc version. I am closing the issue.

[ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 550776 5948528 ./my_test/my_test 32194 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 550776 5948528 ./my_test/my_test 32198 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 550776 5948528 ./my_test/my_test 32203 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 572944 6014064 ./my_test/my_test 32264 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 572944 6014064 ./my_test/my_test 32278 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 572944 6014064 ./my_test/my_test 32282 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 572944 6014064 ./my_test/my_test 32298 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 572944 6014064 ./my_test/my_test 32303 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 572944 6014064 ./my_test/my_test 32308 ali.kel+ 968 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 572944 6014064 ./my_test/my_test 32312 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 572944 6014064 ./my_test/my_test 32317 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 453716 5882992 ./my_test/my_test 32321 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 453716 5882992 ./my_test/my_test 32326 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 573036 6014064 ./my_test/my_test 32365 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 573036 6014064 ./my_test/my_test 32370 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 573036 6014064 ./my_test/my_test 32374 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 573036 6014064 ./my_test/my_test 32378 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 573036 6014064 ./my_test/my_test 32383 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 573036 6014064 ./my_test/my_test 32387 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 573036 6014064 ./my_test/my_test 32391 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 573036 6014064 ./my_test/my_test 32407 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 573036 6014064 ./my_test/my_test 32411 ali.kel+ 968 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 440400 5882992 ./my_test/my_test 32427 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 440400 5882992 ./my_test/my_test 32431 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 440400 5882992 ./my_test/my_test 32436 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 440264 5882992 ./my_test/my_test 32491 ali.kel+ 972 332 grep --color=auto my_test [ali.keles@server17 hazelcast-cpp-client]$ ps -o pid,user,rss,size,command ax | grep my_test 31708 ali.kel+ 440264 5882992 ./my_test/my_test 32495 ali.kel+ 972 332 grep --color=auto my_test