dash
dash copied to clipboard
Set default window type to non-shared static
This PR changes the build scripts and CMakeList to configure DART to use regular windows instead of dynamic + shared memory windows in dart_team_memalloc_aligned.
Background: The use of dynamic windows causes all sorts of troubles:
- It prevents the use of RDMA capabilities on any decent hardware.
- Open MPI chokes if
MPI_THREAD_MULTIPLEis enabled and no RDMA-capable network is found (thanks to @bertwesarg for bringing this back up) because the shared-memory component does not support dynamic windows (how could it?) and thept2ptcomponent lacks support forMPI_THREAD_MULTIPLE(not sure this will ever be fixed). Hence, DASH won't even work on a single node.
The MPI implementation will likely short-cut through shared memory for on-node communication so the benefit of doing it ourselves really seems marginal. On the other hand, changing the default avoids some bad surprises.
Codecov Report
Merging #639 into development will decrease coverage by
0.02%. The diff coverage isn/a.
@@ Coverage Diff @@
## development #639 +/- ##
===============================================
- Coverage 84.94% 84.92% -0.03%
===============================================
Files 335 335
Lines 24821 24821
Branches 11236 11709 +473
===============================================
- Hits 21085 21080 -5
Misses 3733 3733
- Partials 3 8 +5
| Impacted Files | Coverage Δ | |
|---|---|---|
| dash/include/dash/Team.h | 87.36% <0%> (-5.27%) |
:arrow_down: |
I would not consider the build*.sh scripts a good place to document CMake variables. why not inside CMakeLists.txt into the options()?
Btw, does other combinations instead of off&off or on&on make sense too?
I would not consider the build*.sh scripts a good place to document CMake variables. why not inside CMakeLists.txt into the options()?
We can move it to the options in the CMake file. So far these descriptions are kept quite short though and the description of these variables would have to be split into two.
Btw, does other combinations instead of off&off or on&on make sense too?
There is one case where dynamic windows without shared windows make sense. That is related to #615: dynamic windows do not register the memory and thus don't fault the pages, leading to better NUMA behavior. There are also some MPI implementations that don't really do well with shared memory window IIRC. So as long as we don't have the NUMA issue under control I'd leave the two options separate but we should consider merging them at some point.