dragonfly icon indicating copy to clipboard operation
dragonfly copied to clipboard

Dragonfly not rejecting inserts after reaching maxmemory

Open adiholden opened this issue 9 months ago • 4 comments

How to reproduce: run dragonfly ./dragonfly --alsologtostderr --dbfilename= --port=6379 --maxmemory=350MB --proactor_threads=1

run memtier memtier_benchmark -c 2 -t 4 --pipeline=30 --hide-histogram --test-time=3000 --distinct-client-seed --expiry-range=100-10000 --data-size-range=3000-4000

expect dragonfly to reject inserts after reaching 350MB but memtier continues to run inserting more entries

adiholden avatar May 01 '24 06:05 adiholden

it looks like fiber_periodic_done_.WaitFor(period_ms) does not return from wait while running memtier

adiholden avatar May 05 '24 17:05 adiholden

I can confirm I have had the same issue. It behaves very strange when reaching maxmemory.

It always accepts write, and in many cases it seems like it works fine to also get the data so it actually stored it. But in many other cases it does not store the write. So my health check that was based on writing a key, get the value, and compare it with what I wrote actually passed in most cases but for our users it still did not work.

Might be dependant on the size of data written? If it's very close to hitting the limit so some write is actually stored and some are not. But both without error on SET.

Don't know if this information helps, but I hope so.

cyppe avatar May 11 '24 13:05 cyppe

@cyppe if you use cache_mode=true, then Dragonfly will evict entries once it reaches maxmemory.

romange avatar May 11 '24 19:05 romange

Yeh thanks, but I use it not only for cache so auto eviction is not an option here.

Would be better if it rejects operations similar to redis if noeviction is used and it reached maxmemory.

cyppe avatar May 11 '24 20:05 cyppe