Petr Tůma
Petr Tůma
The benchmark has a leaking file descriptor issue. Example exception dump: Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.GeneratedConstructorAccessor3.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at io.netty.channel.ReflectiveChannelFactory.newChannel(ReflectiveChannelFactory.java:38) ... 172 more Caused by: io.netty.channel.ChannelException: io.netty.channel.unix.Errors$NativeIoException:...
Seen more instances of this on regular machines (e.g. measly 8 core 64GB RAM). At the time of the crash, the benchmark has quite a lot of TCP sockets open....
... also see constantly increasing number of threads (right now ~300 threads at iteration 400), if it helps, most thread instances are named UnboundedFuturePool-Something.
... seeing how our Spark instance is hardcoded to run locally (https://github.com/renaissance-benchmarks/renaissance/blob/510b3b9e8f01dc397f35b64a7b5f8a943b75d012/benchmarks/apache-spark/src/main/scala/org/renaissance/apache/spark/SparkUtil.scala#L61) I'm not sure what we're trying to do with the worker instance count ?
I think in local mode there is always only one (in process) `Executor` (with one thread pool) created here https://github.com/apache/spark/blob/7ce7aa47585f579a84dc6dd8f116a48174cba988/core/src/main/scala/org/apache/spark/SparkContext.scala#L2873 and here https://github.com/apache/spark/blob/7ce7aa47585f579a84dc6dd8f116a48174cba988/core/src/main/scala/org/apache/spark/scheduler/local/LocalSchedulerBackend.scala#L62. We can only tweak the number of...
Adding a whole bunch of plots showing how performance depends on the number of threads for individual benchmarks, collected on dual socket Xeon Gold 6230 (2 packages, 20 cores per...
Assuming other machines behave similarly, I think we should cap the number of threads used as follows (with a warning if more cores are available): - 2 or slightly more...
This was measured on Xeon E5-2620 v4 with hyperthreading turned off, so the JVM saw 8 processors, which is the default philosopher count for the benchmark.