kombu icon indicating copy to clipboard operation
kombu copied to clipboard

Allow option to re-queue tasks to front of queue when using brokers that support it.

Open mk-at opened this issue 2 years ago • 5 comments

In our specific application of Celery, it is preferable to not requeue tasks after failure or visibility timeout to the back of the queue as this can cause us to miss SLAs when the queue is large. It would be great if for Redis under broker_transport_options we could specify that we want tasks requeued to the front.

For visibility timeouts, I believe the core functionality to do so is already built in to the Redis client in kombu, we just need to make sure we keep leftmost set to False.

For retries, we can check whether the messages is being retried here, and use rpush instead of lpush if the option is specified to redeliver to the front.

mk-at avatar Jun 06 '23 01:06 mk-at

Hey @mk-at :wave:, Thank you for opening an issue. We will get back to you as soon as we can. Also, check out our Open Collective and consider backing us - every little helps!

We also offer priority support for our sponsors. If you require immediate assistance please consider sponsoring us.

can you come with a PR with appropriate tests?

auvipy avatar Jun 06 '23 03:06 auvipy

I tried to make a branch and push the code, but got access denied. Is there something I need to do to enable to push branches to this repo?

mk-at avatar Jun 06 '23 18:06 mk-at

I have the code all figured out and the test made, just not sure how to push my branch and open a PR

mk-at avatar Jun 06 '23 21:06 mk-at

you first have to fork the repo, then clone it in you pc, then create a branch on your forks clone and push the changes to your newly created branch. and then you will be able to send a PR. you can google how to fork and create a PR on github

auvipy avatar Jun 07 '23 11:06 auvipy