devito
devito copied to clipboard
SparseFunctions + MPI + coordinates
Assume rank0, out of 4 MPI ranks, has 8 items
a b c d e f g h
Now assume that, upon Operator application, the sparse points get distributed via _dist_scatter
to the 4 ranks and the result is
a c --- b d g --- e f --- h
Now, upon returning from Operator application, via _dist_gather
, the sparse points get stored
either as
a c b d g e f h
OR
a b e h c d f g
(to be double checked)
So the layout is different than it was upon data allocation/initialization.
Instead of making sure the layout is deterministically the same, currently the coordinates
are rather reshuffled the same way the sparse points are, so that the mapping position -> value
is correct
This is horrible. Possible approaches to improve this:
- Fix the bug such that upon gather, the layout is as expected
- Make SparseFunctions unordered collections (difficult because we do wanna use
Data
, I imagine...)