incubator-horaedb-meta icon indicating copy to clipboard operation
incubator-horaedb-meta copied to clipboard

Reduce the number of operations written to the etcd storage in a single transaction

Open ZuLiangWang opened this issue 2 years ago • 0 comments

In current etcd storage implementation, it will write and query the metadata of the cluster and shard with transaction containing multiple operations, like this:

	for _, shardView := range req.ShardViews {
		opCreateShardTopologiesAndLatestVersion = append(opCreateShardTopologiesAndLatestVersion, clientv3.OpPut(key, string(value)), clientv3.OpPut(latestVersionKey, fmtID(shardView.Version)))
	}
	resp, err := s.client.Txn(ctx).
		If(keysMissing...).
		Then(opCreateShardTopologiesAndLatestVersion...).
		Commit()

An error occurs when the number of operations exceeds the maximum limit of etcd. Try to optimize these writing methods to reduce the number of operations under the same data volume.

ZuLiangWang avatar Dec 16 '22 06:12 ZuLiangWang