incubator-horaedb-meta
incubator-horaedb-meta copied to clipboard
Reduce the number of operations written to the etcd storage in a single transaction
In current etcd storage implementation, it will write and query the metadata of the cluster and shard with transaction containing multiple operations, like this:
for _, shardView := range req.ShardViews {
opCreateShardTopologiesAndLatestVersion = append(opCreateShardTopologiesAndLatestVersion, clientv3.OpPut(key, string(value)), clientv3.OpPut(latestVersionKey, fmtID(shardView.Version)))
}
resp, err := s.client.Txn(ctx).
If(keysMissing...).
Then(opCreateShardTopologiesAndLatestVersion...).
Commit()
An error occurs when the number of operations exceeds the maximum limit of etcd. Try to optimize these writing methods to reduce the number of operations under the same data volume.