backend.ai
backend.ai copied to clipboard
feat: Introduce Raftify and `RaftGlobalTimer` to replace `DistributedGlobalTimer`
This PR aims to resolve the distribution locking issues by integrating Raftify with Backend.AI manager (based on GlobalTimer operating by Raft algorithm).
Any kind of feedback is welcome.
This PR also partially resolves https://github.com/lablup/backend.ai/issues/1634
How to setup test environment
- Set
num-procofmanager.tomlto an arbitrary number other than 1, and setraftsection when running Backend.AI manager. - Create
raft-cluster-config.tomland setinitial_peersthere. Below is an example.
[[peers.other]]
host = "192.168.0.1"
port = 60151
node-id = 1
role = "voter"
[[peers.other]]
host = "192.168.0.1"
port = 60152
node-id = 2
role = "voter"
[[peers.other]]
host = "192.168.0.1"
port = 60153
node-id = 3
role = "voter"
[[peers.myself]]
host = "192.168.0.2"
port = 60154
node-id = 4
[[peers.myself]]
host = "192.168.0.2"
port = 60155
node-id = 5
Testing and debugging
For putting a new log entry,
curl -XGET http://localhost:60251/put/1/test
For printing all persisted logs,
./backend.ai mgr raft debug persisted-all ./logs