taskiq-redis icon indicating copy to clipboard operation
taskiq-redis copied to clipboard

Disable automatic redelivery for long-running RedisStreamBroker tasks?

Open smalyu opened this issue 3 months ago • 0 comments

We have a cron-like task that scans a large Redis dataset and takes ~35 minutes to complete. With RedisStreamBroker, the default idle_timeout is 600_000 ms (10 minutes). Once the worker runs longer than that, TaskIQ calls XAUTOCLAIM, redelivers the message, and a second instance starts (ours cancels itself via a custom lock, but it still creates noise).

Right now, the only workarounds we see are:

  1. Split the long task into shorter pieces.
  2. Set idle_timeout to a huge value (we currently use one year in milliseconds).

Questions:

  • Is there any supported way to completely disable automatic redelivery, so a job is never re-queued while its worker is alive?
  • If disabling isn’t possible, what’s the recommended approach for genuinely long-running jobs?
  • Was the fixed redelivery behavior chosen mainly for resilience against crashed workers, or do you expect users to handle long tasks differently?

Thanks!

smalyu avatar Oct 01 '25 17:10 smalyu