rocksdb-sharp icon indicating copy to clipboard operation
rocksdb-sharp copied to clipboard

Possible memory leak in rocksdb_iter_key_string

Open adsk-nns opened this issue 1 year ago • 0 comments

Hello,

I use the following code to enumerate all keys in my database. These keys are ASCII strings, either hashes or simple words such as "meta0", "meta1", etc.

public IEnumerable<string> KVGetKeys()
{
    using var iter = _rocksDb.NewIterator();

    var keys = new HashSet<string>();
    iter.SeekToFirst();
    while (iter.Valid())
    {
        keys.Add(iter.StringKey());
        iter.Next();
    }

    return keys;
}

I then enumerate over the returned set while removing some entries from the database. I do this to remove any entries present which I no longer want to keep. I've noticed steady memory growth, and it looks like more Strings are allocated in memory but never freed by the .NET GC.

If I use the following code, any strings created from the byte[] of Key() are freed up and I no longer see any lasting memory growth.

public IEnumerable<string> KVGetKeys()
{
    using var iter = _rocksDb.NewIterator();

    var keys = new HashSet<string>();
    iter.SeekToFirst();
    while (iter.Valid())
    {
        keys.Add(Encoding.ASCII.GetString(iter.Key()));
        iter.Next();
    }

    return keys;
}

Is there something I may have missed? Or are strings allocated by rocksdb_iter_key_string, which iter.StringKey() calls, potentially leaking memory?

adsk-nns avatar Sep 11 '24 17:09 adsk-nns