luceneutil
luceneutil copied to clipboard
Run nightly benchmarks in memory-constrained fashion?
We're currently working on better parallelizing I/O for Lucene queries. To be able to track improvements (and regressions!), it would be nice if we could run nightly benchmarks with a page cache size that is only a fraction of the total index size.
For now we can use ramhog for this if needed. In the future we could specify cgroupv2 limits in the systemd service/timer for all the benchmarks, or run particular benchmarks with systemd-run, specifying lower limits.
@rmuir also suggested using fincore --output-all /path/to/index/* to monitor how many hot/cold pages we see in the index while/after benchmarking.