taskiq-redis
taskiq-redis copied to clipboard
Disable automatic redelivery for long-running RedisStreamBroker tasks?
We have a cron-like task that scans a large Redis dataset and takes ~35 minutes to complete.
With RedisStreamBroker, the default idle_timeout is 600_000 ms (10 minutes).
Once the worker runs longer than that, TaskIQ calls XAUTOCLAIM, redelivers the message, and a second instance starts (ours cancels itself via a custom lock, but it still creates noise).
Right now, the only workarounds we see are:
- Split the long task into shorter pieces.
- Set
idle_timeoutto a huge value (we currently use one year in milliseconds).
Questions:
- Is there any supported way to completely disable automatic redelivery, so a job is never re-queued while its worker is alive?
- If disabling isn’t possible, what’s the recommended approach for genuinely long-running jobs?
- Was the fixed redelivery behavior chosen mainly for resilience against crashed workers, or do you expect users to handle long tasks differently?
Thanks!