Michael McCandless
Michael McCandless
These are surprisingly/depressingly noisy results. Are you sure the are A/A? Exactly same `git clone` of Lucene 9 being compared to itself? Which JVM / Java CL flags are you...
+1. The benchmark is really cheating today by init'ing all the queries up front, and then only measuring how long those take to execute against the index. Changing it to...
> And maybe, just maybe, we have some giant frogs boiling for years that nobody never noticed yet LOL
I chatted w/ @rmuir and he suggested we start simpler here with Java Flight Recorder (JFR). We can later upgrade to `perf`. Progress not perfection! Some notes from our chat:...
I have a start at this! Still iterating on it ... it produces output like this, after all 20 JVM iterations of a benchmark run, when running on `wikimediumall`: ```...
I pushed JFR for searching (still need to enable for indexing too), and it ran last night for the first time: https://home.apache.org/~mikemccand/lucenebench/2021.01.12.00.03.16.html The results are ... curious. E.g. 39% of...
OK I re-ran the `*.jfr` aggregation, asking for `stacksize=10`. I think we really must fix our grouping implementation to not box/unbox `Integer` on every hit! ``` PERCENT HEAP SAMPLES STACK...
And CPU: ``` PERCENT CPU SAMPLES STACK 3.79% 232348 org.apache.lucene.util.packed.DirectMonotonicReader#get() at org.apache.lucene.codecs.lucene80.Lucene80DocValuesProducer$15#binaryValue() at org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts#countAll() at org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts#() at perf.SearchTask#go() at perf.TaskThreads$TaskThread#run() 3.55% 217460 org.apache.lucene.codecs.lucene80.Lucene80DocValuesProducer$15#binaryValue() at org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts#countAll() at org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts#() at perf.SearchTask#go() at...
Good point -- that is a one-time cost, amortized over the life of a service that then handles query traffic. Can I somehow instruct JFR to NOT profile at startup,...
Last night's benchmarks ran with indexer profiling too: ``` Profiler for cpu PROFILE SUMMARY from 471699 events (total: 471699) tests.profile.mode=cpu tests.profile.count=30 tests.profile.stacksize=5 tests.profile.linenumbers=false PERCENT CPU SAMPLES STACK 6.72% 31682 org.apache.lucene.util.ByteBlockPool#setBytesRef()...