spring-integration icon indicating copy to clipboard operation
spring-integration copied to clipboard

Foundation utilities to support the CDC outbox-pattern

Open tzolov opened this issue 2 years ago • 3 comments

As part of your business logic, you often need to create/update/delete your local data store and simultaneously send messages/events to a message broker. Furthermore you need to guarantee the atomicity of both operations (e.g. the message can be sent only if the database transaction is successful). The (2PC) is not a scalable and often not supported option for most distributed systems.

The outbox pattern provides an approach for letting services execute these two operations in a safe and consistent manner.

Spring Integration can provide common utilities to facilitate the creation and management of Outbox table and entities as well as common outbox messaging abstraction.

References:

tzolov avatar Aug 10 '23 17:08 tzolov

Pushed to the next 6.3.x since the change is too involving to make it available for upcoming 6.2.0-RC1 next Tuesday.

artembilan avatar Oct 13 '23 17:10 artembilan

Can you illuminate me on why this can't be achieved with a JdbcChannelMessageStore-backed QueueChannel?

gdupontf avatar Nov 20 '23 20:11 gdupontf

See the pattern page: https://microservices.io/patterns/data/transactional-outbox.html.

In the end there are a couple links like this:

There are two patterns for implementing the Message relay:

The [Transaction log tailing](https://microservices.io/patterns/data/transaction-log-tailing.html) pattern
The [Polling publisher](https://microservices.io/patterns/data/polling-publisher.html) pattern

So, the QueueChannel is based on a DB polling approach, while Debezium uses transaction log tailing. Therefore fully different implementation we are pursuing with this issue unlike the mentioned solution via QueueChannel: https://github.com/artembilan/microservices-patterns-spring-integration/tree/main/outbox

artembilan avatar Nov 20 '23 20:11 artembilan