Handle `PROC_NULL` in a consistent way

[luraess]

Handle MPI.Consts.MPI_PROC_NULL[] in a similar fashion to MPI_COMM_TYPE_SHARED (related / by analogy to #584).

In short, avoid exposing the MPI.Const.XYZ[] to the user.

Note: feel free to edit this issue as the terminology may not be most accurate.

[simonbyrne]

we could just use nothing as a sentinel for PROC_NULL?

[simonbyrne]

I'm not sure this make sense though: from what I can tell, the purpose of MPI_PROC_NULL is that it is an integer sentinel value, so you can store it as an element in an array of ranks: if we use a non-integer value then that won't work?

[luraess]

For consistency, as we have MPI.COMM_WORLD, it would make sense to have MPI.PROC_NULL (as in C - with . instead of _).

[simonbyrne]

What I meant is that MPI.COMM_WORLD is always an opaque object (MPI.Comm in Julia, MPI_Comm in C), so they're treated consistently.

The purpose of MPI_PROC_NULL is that it is the same type as other ranks (i.e. an integer), as the standard says:

In many instances, it is convenient to specify a “dummy” source or destination for communication. This simplifies the code that is needed for dealing with boundaries, for example, in the case of a non-circular shift done with calls to send-receive. The special value MPI_PROC_NULL can be used instead of a rank wherever a source or a destination argument is required in a call.

e.g. say you have a mesh of quadrilaterals, and you store the neighboring ranks across each face; then you can use MPI_PROC_NULL whenever it is a boundary, and the sends and recvs will correctly be no-ops.

However if you make it a different type, then you can no longer store it in an array of ints. This necessitates either using an array of union types (incurring a performance penalty) or choosing your own sentinel value, and adding a branch:

for nbr_rank in neighbors
    if nbr_rank >= 0
        MPI.Send(X, comm; rank=nbr_rank)
    else
        MPI.Send(X, comm; rank=MPI.PROC_NULL)  # this i
    end
end

However the call with MPI.PROC_NULL is a no-op anyway, so you could just write it as

for nbr_rank in neighbors
    if nbr_rank >= 0
        MPI.Send(X, comm; rank=nbr_rank)
    end
end

[luraess]

Thanks for the clarification 🙏. I understand your point and agree that in this example, one could simply "skip" the part involving the rank 0 and call to MPI.PROC_NULL.

Else, I have not much insights, but I'd guess that it would be good to have the Julia implementation to behave as one would expect, in order to avoid confusion. Seing from the perspective of existing code that implements calls to MPI Consts one does not want to fully refactor, both for syntax and behaviour.

[simonbyrne]

We could make MPI.PROC_NULL a non-const value that is set to MPI.Const.MPI_PROC_NULL[]. Unfortunately this will mean that it will be un-typed for Julia versions < v"1.8", but that is probably fine?

@vchuravy what do you think?

[luraess]

Can we make it with MPI.PROC_NULL ?