Add benchmarks

[bartle-stripe]

Go makes it really easy to write synthetic benchmarks. It would be nice if we added some to ccache, since right it's hard to now the perf impact (both in terms of CPU or allocations) of an arbitrary PR change.

[karlseguin]

Good idea. I'll try to get to it at some point, but I'd also accept a PR in the meantime.

[miparnisari]

@karlseguin i'm happy to send a PR but i would need some guidance. What kind of benchmarks would be useful?

[karlseguin]

@miparnisari I'm not 100% sure. The cache is relatively feature rich and has a number of configuration knobs. But, the fundamental operations are Get and Set, and testing them under different conditions seems like a good place to start.

Both Get and Set have different "modes". Get may or may not promote an item. Obviously it won't promote if the item isn't found, but for a found item, the frequency of promotion depends on the GetsPerPromote configuration. It would be interesting to see the impact of GetsPerPromote on performance, and it could help catch some performance regression if the promote path would change.

Set can either update an existing item or create a new item. I assume some people have a workload that rarely replaces items, and some people have a workload that have few items that are frequently replaced. So measuring the performance of Set in both cases would be nice.

Finally, the last major thing to consider is GC pressure. Some people are going to run the cache with very little GC pressure (it's almost used like a lazy-loading hashtable). Some people with run the cache with a bit -> a lot of GC pressure.

Writing small focused benchmark can be useful. Like, create a cache with 10_000 values, then benchmark Get (maybe with different GetsPerPromote) where 10% of items aren't in the cache. Or benchmark Set where 50% of values are replacement and 50% are new.

But writing something that mixes Get and Set might be more interesting, because (a) it's more real world and (b) I expect that to have the worst performance (since they're both processed by the single worker goroutine).

Not sure that really helps you ...more rambling than anything I'm afraid.