eventsourcing icon indicating copy to clipboard operation
eventsourcing copied to clipboard

Google Pub/Sub Support

Open kwiesmueller opened this issue 7 years ago • 2 comments

Hey there, I was looking for approaches to build a new system using event sourcing in golang and so far you implementation seemed the most interesting.

A few questions though. Do you think it would be possible to make this compatible with Pub/Sub (namely Google Pub/Sub) and how does a message queue fit into this?

Persisting the Events would then happen through Subscriptions and receiving commands as well. As far as I could see this shouldn't be an issue. I would also be happy to implement this and contribute if you like.

Do you have any experiences with this? Are you currently using this library in any production project?

Cheers!

kwiesmueller avatar May 30 '18 18:05 kwiesmueller

@kwiesmueller,

Sorry for the delay responding:

  • It's easy enough to do - simply implement it as a middleware, and upon commit publish any created events to your target bus (i.e. Google Pub/Sub).

  • This does raise the question of transactionality though, such as where your event gets written but immediately after your publish target is down/unavailable:

    • In real world, I primarily use DynamoDB with Kinesis streams enabled, giving an atomic 'events in store + publish to bus', - then use Lambda to consume the changes and update views. However, knowing not everyone is in the cloud, I do supply an example in the repo with Mongo using the oplog to achieve the same effect.
  • If your primary event store supports a 'tail mechanism' similar to these, implement that as a seperate unit of code/functionality that people can use as part of a consumer process, but if not, then a 'publish after store' model will work for most scenarios.

Using in a lot of projects, mostly personal, but a there's at least three commercial uses that I'm aware of at this point in time.

steve-gray avatar Jun 04 '18 10:06 steve-gray

Okay, cool! Will give the Middleware Idea a look in the next few days, thanks.

I am still not sure on how the entire pattern will match what we would like to do and how to aproach it so some experiments got to get started. Got to find the right mix between plain Event Driven development for communicating between our services and when to really use Event Sourcing for persistence. Still too new to the pattern it self and unsure about mostly everyhting :grimacing:

kwiesmueller avatar Jun 04 '18 17:06 kwiesmueller