Support send/recv on the same rank

[vishwakaria]

Use-case

I have a client and server, each running on a different thread on the same GPU rank. I need to send data from client thread to server thread using gloo's send operation.

Issue

Does gloo support this out-of-the-box? If yes, can you share a simple snippet? I can implement a pipe-based communication between the two threads, but this would be a nice-to-have feature.

From my code examination, Gloo's TCP implementation for the context does not create a TCPPair for a rank with itself. So send/recv on same rank causes seg faults. I have implemented my send functionality following the looks_like_mpi example:

int Send(const void *buf, int count, void* datatype, int dest, int tag) {
    auto ubuf = kContext->createUnboundBuffer(const_cast<void*>(buf), count);
    ubuf->send(dest, tag);
    ubuf->waitSend();
    return 0;
  }