openfoam-adapter
openfoam-adapter copied to clipboard
precice
Allwmake: Use all available procs
We are currently calling wmake -j 4 and asking the user to change it manually, if they need more. This PR is replacing the wmake -j 4 with wmake -j "$(nproc)".
The motivation for not using nproc earlier was that in cases with many cores but little memory (e.g., a VM), this could lead to some memory usage issues. Measuring on a nproc=8 system, I see that the maximum memory usage is 685MB for 4 threads, and the same for 8 threads (I guess almost everything is OpenFOAM libraries we anyway load). This should be reasonable for any modern system. On the other hand, the time benefit is also rather negligible (if statistically significant at all).
In any case, this will also not over-subscribe the CPU by default (e.g., on a sinle-core system), leading to the same or lower compilation time in all cases.
Originally, we had no parallelization in wmake, as this was not supported by older OpenFOAM versions. All the OpenFOAM versions we now test provide the -j option (i.e., no need to comment on that anymore). Potentially OpenFOAM 4 does not support that, but we can handle such few support requests, if any.
TODO list:
- [x] I updated the documentation in
docs/ - [x] I added a changelog entry in
changelog-entries/(create directory if missing)
My motivation is lower after measuring, actually. I initially thought that the effect would be larger.
OpenFOAM uses a boolean -j flag to build with all cores: https://develop.openfoam.com/Development/openfoam/blob/develop/doc/Build.md#compile-openfoam-faster
But I would not like to make any change that would require adapting scripts and documentation. If we add this, we should then revert the default to sequential, which I guess most people will not notice, making the building significantly slower.
So, the real fix would be to use -j without argument and let the user specify an amount if necessary.
Also keep in mind that -j will compile faster on systems with less the 4 cores, which is common when creating VMs and is the default for GitHub-hosted runners
After discussing with @DavidSCN, we decided to keep this open for now and come up with a more polished solution.
Better, safer suggestion: let's just give a message that one can parallelize OpenFOAM builds by setting the WM_NCOMPPROCS variable in their environment (e.g., ~/.bashrc) to the number of processes.