taskiq icon indicating copy to clipboard operation
taskiq copied to clipboard

Task cancelation from middlewares or dependencies.

Open s3rius opened this issue 2 years ago • 11 comments
trafficstars

We need to provide more control over task execution.

Currently we need to add Cancel error, to be able to implement scenario, described in #132.

s3rius avatar May 15 '23 13:05 s3rius

I created a PR https://github.com/taskiq-python/taskiq/pull/128, which can be described as a non-blocking sleep. Can this be useful in the above scenario?

Sobes76rus avatar May 15 '23 15:05 Sobes76rus

Actually, i believe in their case it will be better to implement broker with custom listen method

Sobes76rus avatar May 15 '23 15:05 Sobes76rus

@Sobes76rus Maybe. But what would listen be doing in that case?

I guess, it's not a broker-related thing, because it's an application logic. Because for some tasks, you may don't want to have rate limiting.

s3rius avatar May 15 '23 15:05 s3rius

We can listen multiple queues in broker And limit some of them

Sobes76rus avatar May 15 '23 15:05 Sobes76rus

I mean, in that case rate-limited tasks should be in different queue

Sobes76rus avatar May 15 '23 15:05 Sobes76rus

I guess it's overcomplicated. Because, what if you need to have different limits for 50 different tasks? You need to listen to 50 different queues then? In my opinion this problem might be solved in a more simple way, by giving you ability to cancel task execution from middleware or dependency. And send it again in the queue.

You can get access to broker message in context, and you can kick directly with broker. After the task is sent, you may cancel current, so it's not going to be written in result backend.

s3rius avatar May 15 '23 15:05 s3rius

Then it should be related to Retry not to Cancel

Sobes76rus avatar May 15 '23 15:05 Sobes76rus

Good point. Yes. Maybe we need to do it with state manager, then.

s3rius avatar May 15 '23 15:05 s3rius

I would like to have a feature like NoResult, It can be implemented using exceptions provided in TaskResult, when we implement exceptions serialization feature

Sobes76rus avatar May 15 '23 15:05 Sobes76rus

Like in Celery RejectError and RetryError exceptions

Sobes76rus avatar May 15 '23 16:05 Sobes76rus

I guess now you can use #137 to simulate this behavior. The only thing we need to add is to set custom task-id to kicker.

s3rius avatar May 18 '23 19:05 s3rius