kombu

kombu copied to clipboard

Current versions of Redis support automatic expires of keys. Can message_ttl support be extended to REDIS instances as well?

Hey @jcpunk :wave:, Thank you for opening an issue. We will get back to you as soon as we can. Also, check out our Open Collective and consider backing us - every little helps!

We also offer priority support for our sponsors. If you require immediate assistance please consider sponsoring us.

if thats possible in redis the probably yes. i suggest you to dig related code a bit to get some initial idea

https://redis.io/commands/expire looks like the right command set, but I'm at a loss where to tack that on.....

queue is declared in kombu here:

https://github.com/celery/kombu/blob/db306b7983becbc43c3b423ca4d69ef0d9c2044e/kombu/entity.py#L625-L656

You can see that channel.prepare_queue_arguments() is needed it needs to be added to kombu/transport/redis.py - see the example in mongo implementation (basically the function needs to be same):

https://github.com/celery/kombu/blob/241b5dcff8a7c8ad411e1b325d59e47acfa9e1ed/kombu/transport/mongodb.py#L317-L319

After that you need "just" to implement the logic. You need to add TTL parameter to _new_queue method:

https://github.com/celery/kombu/blob/db306b7983becbc43c3b423ca4d69ef0d9c2044e/kombu/transport/redis.py#L957-L959

See the example of mongo implementation:

https://github.com/celery/kombu/blob/241b5dcff8a7c8ad411e1b325d59e47acfa9e1ed/kombu/transport/mongodb.py#L150-L157