Two phase-commit
Current transaction engine must support new mode - internal WAL must handle transactions in two states: PREPARED / COMMITTED.
First it appends WAL in PREPARED state and waits until it receives confirmation from replicas of the transaction incorporation. When it receives confirmations from 2f replicas that they PREPARED this transaction, it switches the transaction to COMMITTED STATE, which will be propagated to replicas as well. As soon as the transaction reaches committed state it may be incorporated to shared view.
Beside this there should be two new CommitBehavior levels:
- WAIT_FOR_CHANGES_VISIBLE_TO_MAJORITY
- WAIT_FOR_CHANGES_VISIBLE_TO_ALL
That will represent the moment when the client is informed about transaction end (regardles of the moment when the server incorporates transaction to its shared view state - which relates to the maximum number of replicas allowed to fail).