dragonfly icon indicating copy to clipboard operation
dragonfly copied to clipboard

memory usage ~wrong memory usage reporting

Open Sokharev opened this issue 10 months ago • 2 comments

Describe the bug Adding 100k sorted set members with simple increasing counter eats 25MB vs 8.5MB at redis ( zero score for all elements give also x3 mem ) To Reproduce Steps to reproduce the behavior:

  1. Insert 100k records using ZADD TEST SCORE MEMBER with for loop
  2. Query records MEMORY USAGE TEST

Expected behavior Expected to consume memory as redis or less

Environment (please complete the following information):

  • OS: ubuntu 20.04
  • Kernel: Linux work1 5.10.16.3-microsoft-standard-WSL2
  • Containerized: WSL2
  • Dragonfly Version: v1.14.3-ea6b0ca6772e05c251546ec873bf46ef97d5c588

Reproducible Code Snippet

  //Javascript node redis 
  client = await createClient({ url: "redis://localhost" })
    .on('error', err => console.log('Redis Client Error', err))
    .connect();

  for (var i = 0; i < 100_000; i++)
      await client.zAdd( "TEST2", {score : i, value:i.toString()} );
      //or comment upper line and uncomment below line for adding with zero score
      //await client.zAdd( "TEST", {score : 0, value:i.toString()} );

Sokharev avatar Apr 01 '24 06:04 Sokharev

Thanks for reporting this. The bug seems to be in "MEMORY USAGE" function in Dragonfly and not in the sorted set itself.

romange avatar Apr 01 '24 06:04 romange


#!/usr/bin/env python3

from redis import asyncio as aioredis
import asyncio

async def main():
    redis_client = await aioredis.Redis()  # Adjust your Redis connection
    for i in range(100000):
        item = f"item:{i}" 
        await redis_client.zadd("test1", {item: 0})
    print("All items added!")
    await redis_client.close()
    
if __name__ == "__main__":
    asyncio.run(main())

info memory shows:

used_memory:19170216
object_used_memory:60204384
type_used_memory_ZSET:60204384

which is clearly wrong.

romange avatar Apr 01 '24 06:04 romange