CoreNeuron icon indicating copy to clipboard operation
CoreNeuron copied to clipboard

Enable multi-threading tests (MPI+OpenMP) under CI

Open pramodk opened this issue 4 years ago • 3 comments

I think we are currently we running pure MPI tests under CI. We should run MPI+OpenMP tests to catch any threading errors.

pramodk avatar Apr 20 '20 23:04 pramodk

In the CI we build CoreNeuron SoA with OpenMP enabled https://github.com/BlueBrain/CoreNeuron/blob/76026e0b9820d96524ea3dc0e5c1d9f738e1f55a/tests/jenkins/install_coreneuron.sh#L57 and we run the SoA tests with https://github.com/BlueBrain/CoreNeuron/blob/76026e0b9820d96524ea3dc0e5c1d9f738e1f55a/tests/jenkins/run_corenrn.sh#L17. I see though that for some reason when CoreNeuron runs it prints num_omp_thread=1 even if the OMP_NUM_THREADS is set to 2. Maybe that's because we don't run it with --threading? I need to investigate

iomaganaris avatar Apr 21 '20 08:04 iomaganaris

@iomaganaris : IMO we need a test with more concurrency (more than ring). e.g. If we are using 36 cores on Jenkis CI, using 6 MPI ranks each with 6 threads. This can help to detect race condition bit. We can discuss this what should/can do.

pramodk avatar Apr 21 '20 23:04 pramodk

We have CI builds that enable OpenMP (host) threading: https://github.com/BlueBrain/CoreNeuron/blob/d6507e20ffdcf652b07e231347174f8f9c89e32a/.gitlab-ci.yml#L89 and tests that use multiple threads https://github.com/neuronsimulator/nrn/blob/1b76917231a00281e17bac760d013f9a5c13cd95/test/external/ringtest/CMakeLists.txt#L90-L100, but I'm not sure if this is enough.

olupton avatar Aug 25 '22 07:08 olupton