kafka-connect-redis icon indicating copy to clipboard operation
kafka-connect-redis copied to clipboard

Support transactions

Open jaredpetersen opened this issue 4 years ago • 0 comments

Support transactions via MULTI . . . EXEC.

This might be a little funky. We can go two different routes on this:

  • Single Kafka record that contains a list of commands to be performed in a single transaction
  • Treat MULTI and EXEC as any other command

The former is pretty much out of the question in my mind because it requires a complex data type that people would have to transform their commands into.

The latter adheres to the streaming model better and would be simpler to implement. However, I'm not sure how this works with multiple partitions. Connect tasks can be assigned to multiple partitions and we need to make sure that data coming in from other partitions does not mess up our transaction. This might mean creating multiple Redis clients in each task, one for each partition.

Request originally submitted by @bcashier here: #11

jaredpetersen avatar Nov 24 '20 07:11 jaredpetersen