Drain threads for memory cache hit rate analysis

[bitfaster]

trafficstars

MemoryCache enqueues cache maintenance to the thread pool asynchronously. This leads to non-deterministic results in the hit rate tests, depending on whether cache maintenance has run.

ConcurrentLfu tests are run with a foreground scheduler (no background maintenance) to avoid this problem.

To make the tests fair, wait for all thread pool tasks to complete each time there is a MemoryCache read. This achieves stable results with a small penalty.

Result with fix:

Result before fix:

Thread.Sleep(1) increases the test runtime for glimpse from 0.31 secs to about 15 secs - i.e. 4700% slower. Repeated Thread.Sleep(0) is only about 25% slower.

[coveralls]

coverage: 99.164% (+0.08%) from 99.086% when pulling 58b2d9d2285ecebad17c946e54e0cde85977c8a4 on users/alexpeck/hittp into a60f44eaffde6cba99c8b1c7d10c67aa9233d246 on main.