raft-rs icon indicating copy to clipboard operation
raft-rs copied to clipboard

Supporting parallel raft

Open BusyJay opened this issue 4 years ago • 6 comments

Is your feature request related to a problem? Please describe. Paper introduces a mechanism that allows applying raft logs out of order. Quotes from the paper:

. As the I/O depth increases, the performance gap between two protocols becomes wider. When I/O depth grows to 32, the latency of Raft is approximately 2.5X times of the latency of ParallelRaft, with less than half of the IOPS ParallelRaft achieves.

This feature is also useful in TiKV.

  1. TiKV already have memory locks for transaction writes, so the dependency between raft entries are already known.
  2. If we support multi-range in the future, each range is independent by design, which can reduce the latency and even make it possible to scale vertically in single group.

BusyJay avatar Sep 23 '20 09:09 BusyJay

If we support multi-range in the future, each range is independent by design,

What does multi-range mean?

Fullstop000 avatar Sep 24 '20 07:09 Fullstop000

It means single raft group manages multiple ranges.

BusyJay avatar Sep 24 '20 07:09 BusyJay

It means single raft group manages multiple ranges.

Only set up a single raft group but serve the commands and replication for multiple regions?

Fullstop000 avatar Sep 24 '20 07:09 Fullstop000

Yes, exactly.

BusyJay avatar Sep 24 '20 08:09 BusyJay

https://zhuanlan.zhihu.com/p/228335745 may be helpful which provides a TLA+ formal verification, as well as a detailed explanation for the original algorithm.

breezewish avatar Sep 27 '20 04:09 breezewish

https://zhuanlan.zhihu.com/p/228335745 may be helpful which provides a TLA+ formal verification, as well as a detailed explanation for the original algorithm.

I have seen it yesterday. What a coincidence!🤣

gengliqi avatar Sep 27 '20 07:09 gengliqi