rmq icon indicating copy to clipboard operation
rmq copied to clipboard

[Question] How to process unacked tasks?

Open AhmedAbouelkher opened this issue 2 years ago • 7 comments

Hello, I noticed that if the producer is running and the consumer stopped working for some reason when the consumer restarted it doesn't start where it finished, hence some unknown number of messages will be lost and not processed.

I'm currently using prefetchLimit=5 and pollDuration=100 ms with only one consumer using AddConsumerFunc.

Side note:

I tried to view the redis db data using Medis and did not find any entries at any given time.

AhmedAbouelkher avatar Jul 17 '22 10:07 AhmedAbouelkher

Hey, please have a look at the Cleaner section in the Readme :v:

wellle avatar Jul 17 '22 21:07 wellle

I still do not understand fully how the cleaner works. I read the docs and still.

My thoughts

  • Repushes the unawked messages to be consumed again.
  • Removed all unawked and lose them.

AhmedAbouelkher avatar Jul 18 '22 08:07 AhmedAbouelkher

Well you will need to have a cleaner running. It will detect when consumers are no longer alive and will then move back the messages which had been unacked for them back to the ready list, so they can be consumer by other consumers.

wellle avatar Jul 18 '22 08:07 wellle

Well you will need to have a cleaner running. It will detect when consumers are no longer alive and will then move back the messages which had been unacked for them back to the ready list, so they can be consumer by other consumers.

Ok So the cleaner should be running as a separate service/process, right?

AhmedAbouelkher avatar Jul 18 '22 08:07 AhmedAbouelkher

Exactly 👍

wellle avatar Jul 18 '22 08:07 wellle

I tried your idea, and it works.

I have another question, How can I reprocess rejected messages?

AhmedAbouelkher avatar Jul 19 '22 12:07 AhmedAbouelkher

https://github.com/adjust/rmq#return-rejected-deliveries

wellle avatar Jul 19 '22 12:07 wellle