lni
any benchmark against etcd?
I built an etcd clone on top of Dragonboat (not yet released). Performance is comparable to etcd using etcd's basic benchmarking tools and this kubecon-etcd-metrics-lab repo from a kubecon talk 2 years ago.
It's important to remember that Etcd is a distributed database and Dragonboat is a multigroup consensus library so dragonboat can only be compared to etcd-io/raft, not etcd-io/etcd.
One place where the two raft implementations differ is that Etcd's raft implementation supports learners but those learners can't serve reads. Reads can only be served by active voting members in etcd-io/raft. Dragonboat by contrast allows non-voting members to serve both stale and consistent reads.
See Page 37 of the Raft thesis:
The mechanism to support non-voting servers can also be useful in other contexts; for example, it can be used to replicate the state to a large number of servers, which can serve readonly requests with relaxed consistency
This capability hinted at in the thesis is supported by Dragonboat but not by Etcd's raft library. Depending on the architecture of your solution, this feature might enable better performance on dragonboat vs etcd-io/raft.
Benchmarks below are a side by side comparison of ectd v3.5.9 and my etcd clone implemented from scratch in Go on top of dragonboat. Performance tests are from kubecon-etcd-metrics-lab.
etcd
> docker run --network="host" -it --entrypoint benchmark --rm docker.io/bkanivets/etcd:v3.5.9 \
--endpoints=127.0.0.1:2379,127.0.0.1:22379,127.0.0.1:32379 --clients=1000 --conns=1000 \
put --sequential-keys --key-space-size=100000 --total=1000000
1000000 / 1000000 Boooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo! 100.00% 17s
Summary:
Total: 17.7870 secs.
Slowest: 0.0892 secs.
Fastest: 0.0003 secs.
Average: 0.0172 secs.
Stddev: 0.0089 secs.
Requests/sec: 56220.9792
Response time histogram:
0.0003 [1] |
0.0092 [163223] |∎∎∎∎∎∎∎∎∎∎∎∎∎
0.0181 [467893] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
0.0270 [240692] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
0.0359 [87269] |∎∎∎∎∎∎∎
0.0447 [28711] |∎∎
0.0536 [8874] |
0.0625 [1793] |
0.0714 [1095] |
0.0803 [440] |
0.0892 [9] |
Latency distribution:
10% in 0.0080 secs.
25% in 0.0107 secs.
50% in 0.0152 secs.
75% in 0.0214 secs.
90% in 0.0290 secs.
95% in 0.0344 secs.
99% in 0.0460 secs.
99.9% in 0.0666 secs.
dragonboat
> docker run --network="host" -it --entrypoint benchmark --rm docker.io/bkanivets/etcd:v3.5.9 \
--endpoints=127.0.0.1:19000,127.0.0.1:19001,127.0.0.1:19002 --clients=1000 --conns=1000 \
put --sequential-keys --key-space-size=100000 --total=1000000
1000000 / 1000000 Boooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo! 100.00% 17s
Summary:
Total: 17.2778 secs.
Slowest: 0.0749 secs.
Fastest: 0.0006 secs.
Average: 0.0166 secs.
Stddev: 0.0061 secs.
Requests/sec: 57877.5736
Response time histogram:
0.0006 [1] |
0.0080 [42192] |∎∎∎
0.0155 [435692] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
0.0229 [380936] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
0.0303 [112415] |∎∎∎∎∎∎∎∎∎∎
0.0378 [23357] |∎∎
0.0452 [4635] |
0.0526 [570] |
0.0601 [153] |
0.0675 [43] |
0.0749 [6] |
Latency distribution:
10% in 0.0097 secs.
25% in 0.0123 secs.
50% in 0.0158 secs.
75% in 0.0200 secs.
90% in 0.0246 secs.
95% in 0.0278 secs.
99% in 0.0352 secs.
99.9% in 0.0443 secs.
@kevburnsjr can u pls mention the hardware and how many nodes etc? spec of hardware, os and nodes deployed
3 local containers running on a single desktop computer
Windows 10 WSL 2 Ubuntu 22.04
12th Gen Intel(R) Core(TM) i5-12600K
32GB DDR5 @ 5600 Mhz
temporary file system
This is a trivial benchmark. I intend to create a multi-node real workload simulation but not until the code is ready to be released.
@sprappcom if performance is important, make sure you use tan as your logdb implementation.
@sprappcom I will convert it to rust when AI can 100% correctly doing it without any human intervention.
;-)
please see the README.md file
https://github.com/lni/dragonboat/blob/master/internal/tan/README.md