pg-sharding
Atomicity and isolation of execution of meta change commands.
Proposed design of feature:
Atomicity and isolation of execution of meta change commands. The task is MVP before executing a batch of commands in transactional mode. The task only concerns "writing" commands. Commands are divided into 2 groups: The first group is commands that have business logic in the layer below the meta layer. The second group of teams whose entire business logic lies in meta or without changing the current features are brought to them. Example of the first group of commands:
- Split - logic in the DB layer (upd. logic removed form lock)
- Lock/unlock - the logic of retrays in the database (upd. logic removed form lock-unlock)
- Move KEY RANGE- a mix of actions in etcd and shards These commands will remain non-transactional AS IS. At least at the MVP stage.
An example of the command that can be brought to the second group is CreateKeyRange (see the commit below) .
The rest of the commands will be redesigned so that they can be executed atomically. For some non-transactional commands, their transactional counterparts will be implemented. For example, lock/unlock without retrays in the DB layer. The atomicity of executing complex commands will be provided by the etcd CAS mechanism (and its emulation in memdb). 2 keys will be created:
- TransactionID is the transaction ID to provide a mechanism for optimistically blocking transactional mode commands.
- lockOperations is a pessimistic lock that is set at the beginning of a non-transactional operation and lifted at the end. lock/unlock are excluded. The established pessimistic lock will not allow the use of transaction commands. This lock ensures the isolation of transactional commands from non-transactional ones, ensuring the safe execution of the latter in a concurrent mode.
In case of failure during a non-transaction operation, the DROP LOCK_OPERATIONS command will be created for the system administrator. In manual mode, it will allow you to remove the pessimistic lock.
Transactional commands will be executed in 2 phases:
- The prepare phase returns a chunk, a set of database commands and gossip commands for routers, to the meta layer for each sub-command.
- Application phase: all chunks are assembled sequentially into one chunk and sent to the coordinator for application in one operation. The coordinator updates the database through CAS and, if successful, sends gossip commands to the routers. If a sub-command is needed in a non-transactional operation, it will be replaced by 2 consecutive actions. For example: kr, err := mngr.CreateKeyRange(...) will be transformed: chunk, err := mngr.CreateKeyRange(...) kr, err:= mngr.ExecNoTran(chunk)
Trial implementation: The PoC branch (https://github.com/xelavopelk/spqr/tree/mtga_v2 ). The amount of work on modifying one subcommand into transactional mode can be estimated by the commit: https://github.com/xelavopelk/spqr/pull/1/commits/9e59f02fa76c3f4be7fb94c98384e8ac47def0a3 (without examples and docker).
LIMITATIONS: There is default etcd limit of batch 128 commands. The default grpc command size 1.5 Mb. There is no a critical limit on the stage of atomic commands. It will bi critical for multicommand scripts.
