opengrok icon indicating copy to clipboard operation
opengrok copied to clipboard

Default max number of search threads seems unfounded

Open vladak opened this issue 8 years ago • 2 comments

Configuration class constructor sets the max number of search threads to: 2 * Runtime.getRuntime().availableProcessors(). It's hard to say on what the formula is based (the original formula had +2 which I removed in the changes for #1119) - whether it tries to account for CPUs with HyperThreading (where the OS would display just the cores). Currently both Linux and Solaris return the full count of hardware threads from sysconf(_SC_NPROCESSORS_ONLN)) or if it tries to reflect some Lucene internal behavior or something else.

It would be good to perform some benchmarking on various HW platforms and configurations to see if the overcommit is really needed.

vladak avatar Jun 22 '16 12:06 vladak

originally I added this number based on max threads on a system for such load It's definitely better to ask for full count of hw threads from java ... (jni?)

tarzanek avatar Jun 23 '16 14:06 tarzanek

My guts say, 2 + Runtime.getRuntime().availableProcessors() should be enough. However, today most instances probably run in a zone[like environment], which is not the only one wrt. the bare metal box. So if og causes too much stress, the admin needs to configure this param anyway...

Using Runtime.getRuntime().availableProcessors() is IMHO sufficient, because this reports all available strands/virtual CPUs the JVM is able to use, i.e. no need to use something else (which should provide what?).

jelmd avatar Nov 19 '16 21:11 jelmd