quorum icon indicating copy to clipboard operation
quorum copied to clipboard

[WIP] Make Raft id deterministic, remove required `--raftjoinexisting $raftId`

Open libby opened this issue 3 years ago • 2 comments

The purpose of this PR is to address issue 1063 https://github.com/ConsenSys/quorum/issues/1063 "Raft: Modify to generate deterministic raftIds, remove required --raftjoinexisting $raftId flag " to simplify adding / removing and managing nodes in a raft Quorum network.

Current Raft Flow Adding a Node (before this PR)

The current raft id is a monotonically increasing number that is calculated initially by the index of the nodes in the initial static-nodes.json file. The raft id being tied to the node's position in the static-nodes.json only holds when the network is being started for the first time, after that, to add a new node to a quorum network the nodes must:

  1. be added to a permisioned-nodes.json file on a cluster node.
  2. an operator must run raft.addPeer(enodeurl) on a healthy node in the cluster and obtain the returned id (they cannot know this ahead of time).
  3. the flag --raftjoinexisting $RAFTID must be included in the start up params of the node being added, and the id obtained from the operator must be set in the startup params.

The raftId for a node is not sticky, and if a raft node is removed from the cluster and re-added it must obtain a new raftId. Raft id are never reused to avoid race-conditions, and to compute a unique id when adding other nodes. This divergence of startup parameter makes make it difficult to mange a raft quorum network, e.g. from projects like qubernetes, and from other ops standpoints, as the node parameters as constantly changing, and values have to be obtained outside of the node before starting it up.

This PR makes the raftId deterministic, specifically it changes the raftid from a monotonically increasing uint16 to a deterministic uint64 Keccack hash of the enode id of each node.

Changes To Raft Id / Raft Flow

This PR changes the way that the raft id is calculated and greatly simplifies adding / removing and managing raft quorum nodes / network.

  • Each node can now calculate its own raft id as it the Keccack hash of its enode id.
  • When a node is removed and re-added it will keep the same raft id.
  • The order of the nodes (when initializing a quorum network) in the static-nodes.json is no longer important, as the order does not play a role in the calculating the raft id.
  • Removes the need for the --raftjoinexisting $raftid startup parameter for nodes that are being added to the network.
  • All nodes can have the same geth startup parameters, there is no difference between the params of an initial node and subsequent nodes being added to the network.
  • Collision are avoided due to the hash properties although it is not 256 but uint64, it should suffice for a reasonable sized raft network up to thousands of nodes.

libby avatar Sep 08 '20 15:09 libby

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

CLAassistant avatar Feb 23 '22 09:02 CLAassistant

What happened to this feature ? We were very much looking forward to it!

robinbryce avatar Aug 15 '22 11:08 robinbryce