bigcache-bench icon indicating copy to clipboard operation
bigcache-bench copied to clipboard

Move benchmark-unrelated code out of the hot path

Open rockdaboot opened this issue 1 year ago • 0 comments

As the title says. It should also obsolete #16. Every string instantiation in key(i) caused a heap allocation, which tainted the benchmarks.

[UPDATE] Meanwhile I added some more commits to move all code out of the hot path that is not directly related to benchmarking set/get. E.g. creation of keys and values including random number generation should not add up to the measurements of the cache API (except for ser/deser code that some cache APIs require while others don't require it or do it internal).

Before

$ go version
go version go1.21.4 linux/amd64
goos: linux
goarch: amd64
pkg: github.com/allegro/bigcache-bench
cpu: 12th Gen Intel(R) Core(TM) i7-12800H
BenchmarkMapSetForStruct-20                         4574           1050507 ns/op          697285 B/op      19746 allocs/op
BenchmarkSyncMapSetForStruct-20                     1512           3062503 ns/op         1828855 B/op      69662 allocs/op
BenchmarkOracamanMapSetForStruct-20                 3039           1593436 ns/op         1216733 B/op      20144 allocs/op
BenchmarkFreeCacheSetForStruct-20                   1172           4117920 ns/op         6987192 B/op      40287 allocs/op
BenchmarkBigCacheSetForStruct-20                    1362           3640569 ns/op         3771568 B/op      42459 allocs/op
BenchmarkMapSetForBytes-20                          2886           1722816 ns/op         2097688 B/op      29749 allocs/op
BenchmarkSyncMapSetForBytes-20                      1399           3438202 ns/op         3112061 B/op      79919 allocs/op
BenchmarkOracamanMapSetForBytes-20                  2138           2121852 ns/op         2943254 B/op      30175 allocs/op
BenchmarkFreeCacheSetForBytes-20                    1424           3446344 ns/op         7867129 B/op      30537 allocs/op
BenchmarkBigCacheSetForBytes-20                     1602           3034032 ns/op         4651572 B/op      32713 allocs/op
BenchmarkMapGetForStruct-20                     38352931               111.5 ns/op            23 B/op          1 allocs/op
BenchmarkSyncMapGetForStruct-20                 35730880               133.3 ns/op            23 B/op          1 allocs/op
BenchmarkOracamanMapGetForStruct-20             36561675               130.1 ns/op            23 B/op          1 allocs/op
BenchmarkFreeCacheGetForStruct-20                9007900               525.5 ns/op           271 B/op          8 allocs/op
BenchmarkBigCacheGetForStruct-20                 9274410               515.5 ns/op           287 B/op          9 allocs/op
BenchmarkMapGetForBytes-20                      42340047               111.4 ns/op            23 B/op          1 allocs/op
BenchmarkSyncMapGetForBytes-20                  35446474               135.0 ns/op            23 B/op          1 allocs/op
BenchmarkOracamanMapGetForBytes-20              34858657               135.3 ns/op            23 B/op          1 allocs/op
BenchmarkFreeCacheGetForBytes-20                22743499               207.3 ns/op           135 B/op          2 allocs/op
BenchmarkBigCacheGetForBytes-20                 25927096               186.1 ns/op           151 B/op          3 allocs/op
BenchmarkSyncMapSetParallelForStruct-20          8432505               687.5 ns/op            72 B/op          5 allocs/op
BenchmarkOracamanMapSetParallelForStruct-20     113340970               42.98 ns/op           30 B/op          2 allocs/op
BenchmarkFreeCacheSetParallelForStruct-20       84426818                54.65 ns/op           53 B/op          4 allocs/op
BenchmarkBigCacheSetParallelForStruct-20        62022183                74.63 ns/op          222 B/op          4 allocs/op
BenchmarkSyncMapSetParallelForBytes-20           7928991               647.5 ns/op           200 B/op          6 allocs/op
BenchmarkOracamanMapSetParallelForBytes-20      88186278                55.89 ns/op          143 B/op          3 allocs/op
BenchmarkFreeCacheSetParallelForBytes-20        84665589                52.74 ns/op          141 B/op          3 allocs/op
BenchmarkBigCacheSetParallelForBytes-20         57134434                90.22 ns/op          506 B/op          3 allocs/op
BenchmarkSyncMapGetParallelForStruct-20         189450904               25.36 ns/op           23 B/op          1 allocs/op
BenchmarkOracamanMapGetParallelForStruct-20     186230124               26.05 ns/op           23 B/op          1 allocs/op
BenchmarkFreeCacheGetParallelForStruct-20       45472926                99.74 ns/op          271 B/op          8 allocs/op
BenchmarkBigCacheGetParallelForStruct-20        43793809               109.5 ns/op           288 B/op          9 allocs/op
BenchmarkSyncMapGetParallelForBytes-20          206990751               23.17 ns/op           23 B/op          1 allocs/op
BenchmarkOracamanMapGetParallelForBytes-20      190093576               25.74 ns/op           23 B/op          1 allocs/op
BenchmarkFreeCacheGetParallelForBytes-20        91603290                48.53 ns/op          135 B/op          2 allocs/op
BenchmarkBigCacheGetParallelForBytes-20         90818000                47.96 ns/op          152 B/op          3 allocs/op
PASS
ok      github.com/allegro/bigcache-bench       194.972s

After

$ go version
go version go1.21.4 linux/amd64
$ go test -bench=. -benchmem -benchtime=4s ./... -timeout 30m
goos: linux
goarch: amd64
pkg: github.com/allegro/bigcache-bench
cpu: 12th Gen Intel(R) Core(TM) i7-12800H
BenchmarkMapSetForStruct-20                     738278595                6.465 ns/op           0 B/op          0 allocs/op
BenchmarkSyncMapSetForStruct-20                 73809016                68.92 ns/op           40 B/op          3 allocs/op
BenchmarkOracamanMapSetForStruct-20             179029155               27.51 ns/op            0 B/op          0 allocs/op
BenchmarkFreeCacheSetForStruct-20               58267056                71.22 ns/op            8 B/op          1 allocs/op
BenchmarkBigCacheSetForStruct-20                43320409               100.3 ns/op           128 B/op          1 allocs/op
BenchmarkMapSetForBytes-20                      145805667               32.41 ns/op          112 B/op          1 allocs/op
BenchmarkSyncMapSetForBytes-20                  41185538               112.0 ns/op           168 B/op          4 allocs/op
BenchmarkOracamanMapSetForBytes-20              73706602                61.97 ns/op          112 B/op          1 allocs/op
BenchmarkFreeCacheSetForBytes-20                45121453               100.9 ns/op           112 B/op          1 allocs/op
BenchmarkBigCacheSetForBytes-20                 29798607               143.3 ns/op           463 B/op          1 allocs/op
BenchmarkMapGetForStruct-20                     917468594                5.063 ns/op           0 B/op          0 allocs/op
BenchmarkSyncMapGetForStruct-20                 350010081               12.55 ns/op            0 B/op          0 allocs/op
BenchmarkOracamanMapGetForStruct-20             229678330               20.87 ns/op            0 B/op          0 allocs/op
BenchmarkFreeCacheGetForStruct-20               51396169                81.19 ns/op           32 B/op          2 allocs/op
BenchmarkBigCacheGetForStruct-20                81617512                58.30 ns/op           32 B/op          2 allocs/op
BenchmarkMapGetForBytes-20                      816561459                5.388 ns/op           0 B/op          0 allocs/op
BenchmarkSyncMapGetForBytes-20                  387219021               14.09 ns/op            0 B/op          0 allocs/op
BenchmarkOracamanMapGetForBytes-20              239492982               20.39 ns/op            0 B/op          0 allocs/op
BenchmarkFreeCacheGetForBytes-20                46738532               111.6 ns/op           136 B/op          2 allocs/op
BenchmarkBigCacheGetForBytes-20                 63684832                81.63 ns/op          136 B/op          2 allocs/op
BenchmarkSyncMapSetParallelForStruct-20         18331375               274.9 ns/op            41 B/op          2 allocs/op
BenchmarkOracamanMapSetParallelForStruct-20     169913868               27.97 ns/op            0 B/op          0 allocs/op
BenchmarkFreeCacheSetParallelForStruct-20       144847314               31.37 ns/op            8 B/op          1 allocs/op
BenchmarkBigCacheSetParallelForStruct-20        96001162                56.98 ns/op          192 B/op          1 allocs/op
BenchmarkSyncMapSetParallelForBytes-20          13546615               351.2 ns/op           170 B/op          4 allocs/op
BenchmarkOracamanMapSetParallelForBytes-20      121414638               37.81 ns/op          112 B/op          1 allocs/op
BenchmarkFreeCacheSetParallelForBytes-20        131029023               36.91 ns/op          112 B/op          1 allocs/op
BenchmarkBigCacheSetParallelForBytes-20         68330320                75.51 ns/op          482 B/op          1 allocs/op
BenchmarkSyncMapGetParallelForStruct-20         1000000000               3.839 ns/op           0 B/op          0 allocs/op
BenchmarkOracamanMapGetParallelForStruct-20     506150209                9.661 ns/op           0 B/op          0 allocs/op
BenchmarkFreeCacheGetParallelForStruct-20       222971008               24.01 ns/op           32 B/op          2 allocs/op
BenchmarkBigCacheGetParallelForStruct-20        348548276               14.01 ns/op           32 B/op          2 allocs/op
BenchmarkSyncMapGetParallelForBytes-20          1000000000               3.824 ns/op           0 B/op          0 allocs/op
BenchmarkOracamanMapGetParallelForBytes-20      495279145                9.952 ns/op           0 B/op          0 allocs/op
BenchmarkFreeCacheGetParallelForBytes-20        172124143               28.57 ns/op          136 B/op          2 allocs/op
BenchmarkBigCacheGetParallelForBytes-20         203385400               24.08 ns/op          136 B/op          2 allocs/op
PASS
ok      github.com/allegro/bigcache-bench       216.286s

rockdaboot avatar Dec 21 '23 09:12 rockdaboot