swiss icon indicating copy to clipboard operation
swiss copied to clipboard

Any benchmark comparison against cockroachdb's swiss map?

Open kolinfluence opened this issue 1 year ago • 1 comments

as titled

kolinfluence avatar Apr 21 '24 09:04 kolinfluence

I just did a bakeoff on my apple silicon (ARM), the results are wildly different! This is using cockroach's own benchmarks, cockroach on the left, dolthub on the right:

Iteration:
BenchmarkMapIter/impl=swissMap/t=Int/len=1048576-12                  4472918       2289718       -48.81%

Int64s:
BenchmarkMapGetHit/impl=swissMap/t=Int64/len=1048576-12              17.1          18.3          +6.83%
BenchmarkMapGetMiss/impl=swissMap/t=Int64/len=1048576-12             15.1          8.03          -46.96%
BenchmarkMapPutGrow/impl=swissMap/t=Int64/len=1048576-12             52900212      52104365      -1.50%
BenchmarkMapPutPreAllocate/impl=swissMap/t=Int64/len=1048576-12      20431517      19296885      -5.55%
BenchmarkMapPutReuse/impl=swissMap/t=Int64/len=1048576-12            18195871      14626880      -19.61%
BenchmarkMapPutDelete/impl=swissMap/t=Int64/len=1048576-12           35.1          83.2          +136.90%

Strings:
BenchmarkMapGetHit/impl=swissMap/t=String/len=1048576-12             40.3          35.4          -12.12%
BenchmarkMapGetMiss/impl=swissMap/t=String/len=1048576-12            21.7          11.5          -47.17%
BenchmarkMapPutGrow/impl=swissMap/t=String/len=1048576-12            99480000      97972653      -1.52%
BenchmarkMapPutPreAllocate/impl=swissMap/t=String/len=1048576-12     66451148      64990448      -2.20%
BenchmarkMapPutReuse/impl=swissMap/t=String/len=1048576-12           63804971      26531076      -58.42%
BenchmarkMapPutDelete/impl=swissMap/t=String/len=1048576-12          96.0          139           +44.45%

The "PutDelete" benchmark deletes each key before putting it back in the map, which is pretty weird, so unless you do a lot of deletes it looks like this library is the clear winner for the time being. Would love to see the same test run on Intel though.

Also implies there may be scope to improve the Delete method here.

ianwilkes avatar Jul 31 '24 03:07 ianwilkes