likwid icon indicating copy to clipboard operation
likwid copied to clipboard

[BUG] No openMP threads when executing a script instead of binary

Open SeiDPierre opened this issue 1 year ago • 1 comments

Describe the bug When running: likwid-perfctr -C 0-4 -M 1 -g ENERGY ./script.sh only one thread is running. (the script.sh is only calling ./bin.x). While if I first export GOMP_CPU_AFFINITY="0-4" and then run: likwid-perfctr -c 0-4 -M 1 -g ENERGY ./script.sh (note the -c here), the 4 threads are running. The difference is not occuring when directly executing a binary.

To Reproduce

  • Execute likwid-perfctr -C 0-4 -M 1 -g ENERGY ./any_omp_binary.x and run likwid-perfctr -C 0-4 -M 1 -g ENERGY ./script_containg_any_omp_binary.bash. Observe the number of threads running with htop.
  • likwid-perfctr -- Version 5.1.1 (commit: 233ab943543480cd46058b34616c174198ba0459)
  • Debian 6.12.15-1 (2025-02-18) x86_64 GNU/Linux
  • Application use OpenMP.
  • No API

out_perfctr.txt out_topology.txt

SeiDPierre avatar Apr 08 '25 07:04 SeiDPierre

Unfortunately not reproducible because none of our systems has LIKWID 5.1.1 anymore.

  • likwid-perfctr -C 0-4 -M 1 -g ENERGY ./script.sh LIKWID reduces the cpuset for script.sh to the hardware threads 0-4, so as soon as it starts, it can only run on the selected 5 HW threads. pid XXX's current affinity mask: 1f
  • likwid-perfctr -c 0-4 -M 1 -g ENERGY ./script.sh LIKWID does not manipulate the cpuset, so the script.sh has all HW threads to run on. pid XXX's current affinity mask: ffffffffffffffffff

You can see the affinity mask by adding taskset -p $$ to your script.sh.

But for me, both approaches work:

$ cat script.sh
#!/bin/bash -l

taskset -p $$
./streamGCC   # McCalpin's STREAM benchmark compiled with GCC
$ likwid-perfctr -C 0-4 -g ENERGY ./script.sh
Number of Threads requested = 5
Thread 0 running on processor 0 ....
Thread 3 running on processor 3 ....
Thread 4 running on processor 4 ....
Thread 1 running on processor 1 ....
Thread 2 running on processor 2 ....
$ OMP_NUM_THREADS=5 GOMP_CPU_AFFINITY="0-4" likwid-perfctr -c 0-4 -g ENERGY ./script.sh
Number of Threads requested = 5
Thread 3 running on processor 3 ....
Thread 4 running on processor 4 ....
Thread 2 running on processor 2 ....
Thread 0 running on processor 0 ....
Thread 1 running on processor 1 ....

TomTheBear avatar Apr 16 '25 10:04 TomTheBear

Did you try it with a more recent LIKWID version? Otherwise, I would close the issue.

TomTheBear avatar Nov 06 '25 13:11 TomTheBear

I'm sorry for the long time answer, likwid was not the issue. You can close the issue.

SeiDPierre avatar Nov 13 '25 10:11 SeiDPierre