Rebus icon indicating copy to clipboard operation
Rebus copied to clipboard

MongoDB based transport

Open SebastianStehle opened this issue 2 years ago • 2 comments

Hi,

I am working on a MongoDB based transport: https://github.com/SebastianStehle/Rebus.MongoDb.Transport/blob/main/MongoTransport.cs

The idea is to have a fallback when my customers do not have anything like Azure Queue or so on and have a small installation.

I use find and modify to ensure that no other worker fetches the message. But what happens when the process crashed after the commit has been completed? Is there something like a timeout header after which the message could be moved back to "unhandled"? e.g. with a timer.

SebastianStehle avatar Jul 01 '22 09:07 SebastianStehle

Actually I've made a message queue implementation for MongoDB as well: https://github.com/rebus-org/MongolianBarbecue

I just never made a Rebus transport for it. But maybe you can get inspiration from it? It uses find-and-modify too, and then it attaches a timestamp to the modified document, effectively locking it for a fixed duration. This way, if the process crashes, the message will pop up as visible again, once the timeout has elapsed.

mookid8000 avatar Jul 23 '22 11:07 mookid8000

This way, if the process crashes, the message will pop up as visible again, once the timeout has elapsed.

Yes, this was also my idea. But this timeout should be defined by the application and I have not seen something like this in Rebus. Ideally it is the same timeout that is used for a single message to be processed. For example Google PubSub also ha a acknowledge timeout: https://cloud.google.com/pubsub/docs/reference/rest/v1/projects.subscriptions/acknowledge

SebastianStehle avatar Jul 25 '22 07:07 SebastianStehle

Rebus.MongoDb will have a new transport implementation again in v8

mookid8000 avatar Nov 14 '23 15:11 mookid8000