dragonfly icon indicating copy to clipboard operation
dragonfly copied to clipboard

`SADDEX` is broken

Open zidokobik opened this issue 4 months ago • 3 comments

Describe the bug SADDEX fails to expire members occasionally. Leaving them not having an expiration.

To Reproduce Here's a test run in python with the redis-py package:

from asyncio import sleep, Runner, gather
from random import random
from time import time_ns
from redis.asyncio import Redis

TIMEOUT = 10
KEY = 'concurrentRequest:user:1'


async def simulate_request(redis):
    member = time_ns()
    await redis.execute_command('SADDEX', KEY, TIMEOUT, member)
    await sleep(random())  # simulate handling the request

    if random() < 0.05: # 5% chance to fail, members should be expired automatically
        raise Exception
    await redis.srem(KEY, member)


async def main():
    # simulate 5000 requests
    async with Redis(decode_responses=True) as redis:
        requests = [simulate_request(redis) for _ in range(5000)]
        await gather(*requests, return_exceptions=True)

        # wait for the remaining failed requests to expire
        await sleep(TIMEOUT + 1) 

        # actively expire by reading the key
        await redis.smembers(KEY)

        # assert the set is empty after running the test
        cardinality = await redis.scard(KEY)
        print(cardinality)
        assert cardinality == 0


if __name__ == "__main__":
    with Runner() as runner:
        runner.run(main())

Expected behavior The test runs without error, i.e. the set should be empty.

Environment (please complete the following information):

  • OS: Debian 11 Bullseye
  • Dragonfly Version: v1.23.2-4959bef8d17e6132b4227ea7ca413faf1b1dc037

Additional context I have a service in which I need to keep track of client's requests concurrency. I use a set to store the request starting-time as members, then it's SREM when the response is returned, therefore keeping the cardinality as the number of on-flight requests. I use SADDEX (add members with expiration to a set) instead of the normal SADD so that if any error occurs, the member will get removed automatically after a certain timeout. I'm aware that SADDEX is dragonfly-only and experimental, it's the one of the reason I'm using dragonfly over redis.

zidokobik avatar Oct 13 '24 20:10 zidokobik