Memory bloat or leak?

[tboyko]

Using gem v 4.0.1 causes RSS memory to continually increase with each access to redis. Perhaps someone can shed some light on this?

Gist demonstrating behavior: https://gist.github.com/tboyko/943f52702fa9a623da56acd19130c141

Output:

start: 17,708 kilobytes
after cleanup: 17,768 kilobytes
after insert: 160,016 kilobytes
after pop: 162,576 kilobytes

[bvirlet]

@tboyko false alert or solved otherwise?

[tboyko]

@bvirlet still seeing the issue but closed the issue because I didn't want to pull in any victims to my own potential memory issue. Are you seeing similar? I'll reopen if so.

I also thought that line 20 may have been the culprit: possibly initializing many many non-heap elements with each iteration that never get GCed. But, I also tried pulling that string initialization out of the loop and reusing it. That did not fix the issue.

[bvirlet]

• Have you tried downgrading to Redis 3.x?
• You should try to use memory_profiler to see what you can reproduce.
• I'm having an issue but I can reproduce it with Redis 3.x so I'm not sure if this is the same issue.

[tboyko]

Yes, seeing the same issue with redis 3.3.5 and MRI ruby 2.4.0 on macOS.

[bvirlet]

You're not seeing a leak, you're just seeing Ruby's memory grow and it ends up plateau-ing.

See the output of your tweaked gist: https://gist.github.com/bvirlet/41b99f256c8412f2518b8d40ca5fb946

start: 12,724 kilobytes
after cleanup: 12,748 kilobytes
after insert: 162,740 kilobytes
after pop: 163,556 kilobytes
start: 163,556 kilobytes
after cleanup: 163,556 kilobytes
after insert: 181,980 kilobytes
after pop: 188,368 kilobytes
start: 188,368 kilobytes
after cleanup: 188,368 kilobytes
after insert: 188,588 kilobytes
after pop: 188,420 kilobytes
start: 188,420 kilobytes
after cleanup: 188,420 kilobytes
after insert: 188,420 kilobytes
after pop: 188,420 kilobytes
start: 188,420 kilobytes
after cleanup: 188,420 kilobytes
after insert: 188,428 kilobytes
after pop: 188,436 kilobytes
start: 188,436 kilobytes
after cleanup: 188,436 kilobytes
after insert: 188,452 kilobytes
after pop: 188,452 kilobytes
start: 188,452 kilobytes
after cleanup: 188,452 kilobytes
after insert: 188,452 kilobytes
after pop: 188,452 kilobytes
start: 188,452 kilobytes
after cleanup: 188,452 kilobytes
after insert: 188,452 kilobytes
after pop: 188,452 kilobytes
start: 188,452 kilobytes
after cleanup: 188,452 kilobytes
after insert: 188,452 kilobytes

Ruby is just requesting enough memory from the system to allocate all the new objects. Once done, it cleans up the object in that memory but doesn't return it to the system. However, if it needs to allocate more objects, they will be allocated in that memory.

To detect leaks you can use https://github.com/SamSaffron/memory_profiler.

[tboyko]

With your same gist, I'm seeing the following (commented out some output for clarity):

after pop: 194,412 kilobytes
after pop: 199,288 kilobytes
after pop: 199,512 kilobytes
after pop: 201,340 kilobytes
after pop: 201,340 kilobytes
after pop: 207,228 kilobytes
after pop: 220,640 kilobytes
after pop: 220,640 kilobytes
after pop: 220,648 kilobytes
after pop: 220,648 kilobytes
after pop: 221,364 kilobytes
after pop: 221,364 kilobytes
after pop: 229,212 kilobytes
after pop: 229,212 kilobytes
after pop: 235,544 kilobytes
after pop: 237,408 kilobytes
after pop: 237,408 kilobytes
after pop: 237,440 kilobytes
after pop: 229,272 kilobytes
after pop: 229,944 kilobytes
after pop: 224,836 kilobytes
after pop: 229,948 kilobytes
after pop: 229,948 kilobytes
after pop: 231,188 kilobytes
after pop: 223,108 kilobytes
after pop: 223,108 kilobytes
after pop: 230,500 kilobytes
after pop: 231,896 kilobytes
after pop: 238,228 kilobytes
after pop: 231,896 kilobytes
after pop: 231,896 kilobytes
after pop: 231,896 kilobytes
after pop: 231,896 kilobytes
after pop: 231,896 kilobytes
after pop: 231,896 kilobytes
after pop: 232,280 kilobytes
after pop: 232,280 kilobytes
after pop: 232,280 kilobytes
after pop: 224,088 kilobytes
after pop: 226,744 kilobytes
after pop: 224,088 kilobytes
after pop: 236,824 kilobytes
after pop: 240,692 kilobytes
after pop: 240,692 kilobytes
after pop: 232,500 kilobytes
after pop: 232,500 kilobytes
after pop: 232,500 kilobytes
after pop: 238,312 kilobytes
after pop: 240,692 kilobytes
after pop: 240,692 kilobytes
after pop: 240,692 kilobytes

It drops down at times but continues to make new highs.

The version of redis server wouldn't affect this, correct? Also, what version of ruby are you using?

[byroot]

The network layer has been entirely rewritten in 5.x, so I'm going to assume this issue no longer applies.

Feel free to comment if you see similar issues on 5.x and I'll reopen.