redpanda icon indicating copy to clipboard operation
redpanda copied to clipboard

Published 20 hours ago • verified publisher icon redpanda-data

Remove prepare phase for transactions and move to acks=all for commits to the tx log

Open mmedenjak opened this issue 2 years ago • 1 comments

Based on these perf findings - https://redpandadata.slack.com/archives/C03CSNSGWKC/p1663340367762939

mmedenjak avatar Sep 20 '22 15:09 mmedenjak

@mmedenjak , would you mind providing a quick summary of the Slack discussion mentioned? As the link doesn't seem to be public.

freef4ll avatar Sep 21 '22 07:09 freef4ll

@VadimPlh @rystsov can you summarise? I feel like I'd do a poor job at it.

mmedenjak avatar Sep 25 '22 19:09 mmedenjak

@VadimPlh @rystsov can you summarise? I feel like I'd do a poor job at it.

Idea is: do not write prepare marker for rm_stm. We did it to sync all prev records for tx. Because we write data_batches with leader ack. And it does not provide that all followers flush thus records. Prepare marker was written with quorum_ack. And after we apply it we can be sure that records for transaction is replicated and flushed.

What we wanna do:

  • Do not write prepare_marker
  • Change leader_ack to quorum_ack for data batches
  • Store tx_seq at the begin_tx because we need it on expire old_tx.

VadimPlh avatar Sep 26 '22 12:09 VadimPlh