Michael McCandless

Results 356 comments of Michael McCandless

> Maybe you could compare performance with Java 23 on the commit before Lucene started requiring Java 24 ([612f0da](https://github.com/apache/lucene/commit/612f0da4a4ce3a133b40402a87ec5cf7eeb290cc)). If performance is still bad, this would indicate that the regression...

Thanks @uschindler! I was able to get @jpountz idea to work -- it ran in last night's run (2025-05-15) and it looks to me like Java 23 -> 24 was...

OK thank you @jpountz and @rmuir! Downgrading the kernel to 6.12.4 indeed recovered our lost performance, so it looks like that change in HZ (how many time slices the Linux...

OK I've re-upgraded everything (Lucene is on tip `main`, Java is back to 24, Linux kernel now even further upgraded to 6.14.6) and kicked off benchy run. @rmuir explained a...

Hmm @yugushihuang (on my team (Amazon product search) team) found this is another way to query the kernel (our Amazon Linux 2 boxes seem not to have `/proc/config.gz`): ``` getconf...

It looks like this `CONFIG_HZ` change also impacted KNN indexing throughput: https://benchmarks.mikemccandless.com/knnResults.html

> It looks like nightly benchmarks only run every 2 days since May 13th, vs. every day before that. Is this because it now takes longer to run the benchmark?...

> A downside with option 1 could be limited control on merge scheduling. Maybe with option 1 we could add a package private API that allows the manager to drop...

OK I like these tradeoffs -- +1 to a new `MergeScheduler` with a fixed thread pool, and starting simple (no intelligence about being "fair" when writers are asking for too...

> > Using the same thread pool for indexing and merging. This way if the thread pool gets full of merges, this will naturally push back on indexing. > >...