dragonfly

dragonfly copied to clipboard

8 threads, default memtier benchmark with pipeline=50

memtier_benchmark --ratio=1:10 -t 8 --pipeline 50 --hide-histogram --test-time 10 --distinct-client-seed

qps in millions based on pipeline squashing

no: 3.0
20: 1.7 !!!???
45: 3.0
55: 3.0

---

I noticed this when running a benchmark, with a large number of clients pipeline squashing is not effective at all, in the second example it's more effective for a smaller number of clients

./bench numer-of-goroutines pipeline-size

squash=0

$ redis-cli debug populate 10000000 key 500
OK
$ ./bench 150 100
1.28438
$ ./bench 50 100
0.94857
$ ./bench 50 200
0.98692

squash=20

$ redis-cli debug populate 10000000 key 500
OK
$ ./bench 150 100
0.83221
$ ./bench 50 100
1.21242
$ ./bench 50 200
1.07138

squash = 50

$ redis-cli debug populate 10000000 key 500
OK
$ ./bench 150 100
1.31743
$ ./bench 50 100
1.04436
$ ./bench 50 200
1.1295

Based on the fact that the cpu usage in the outside case is well below 100%

and no obvious bottleneck are present in the profiles

bench-bad bench-good

I assume this is caused by a bad io pattern (?) because with squashing=20 we might flush twice 🤔