conduit
conduit copied to clipboard
relay mpi 'any-to-any'
While thinking about a mpi all-to-all interface, @bryujin and I talked over several potential interfaces.
The best candidate surviving is one that constructs node trees that include destination ranks.
For example, given an MPI job w/ three ranks, where we want to perform the following exchange:
Rank 0 sends A to Rank 1 and Rank 2
Rank 0 sends B to Rank 2
Rank 1 sends C to Rank 2
Rank 2 sends D to Rank 0
Callers on the ranks would construct input nodes that look like the following:
Rank 0:
[
{ "dest" : [1,2] , "value" : (A)},
{ "dest" : [2] , "value" : (B)}
]
Rank 1:
[
{ "dest" : [2] , "value" : (C)}
]
Rank 2:
[
{ "dest" : [0] , "value" : (D)}
]
set_external() can used to zero-copy the data (with some cost to copy the meta-data)
And the output would look like the following:
Rank 0:
[ (D) ]
Rank 1:
[ (A) ]
Rank 2:
[ (A,B,C) ]
This isn't really a generic all to all interface. (But I belive it can be implemented with an all-to-all). We will have to think about the proper name for this. 'any-to-any' is a place-holder, which may be good enough.
the common need seems to be a recipe for groups of sparse isend irecv pairs, this pattern is more flexible and efficient. Instead of any-to-any, we should provide a clear tutorial on how to implement this pattern with arbitrary payloads