SOS
SOS copied to clipboard
mmap shared memory transport
This new on-node transport mmap's the symmetric data and heap segments of all other PE's, much like XPMEM does. The intent is to provide an alternative to XPMEM for shared memory transport when XPMEM is not available. The RMA performance looks very similar to XPMEM.
To create a file-backed mmap of the data segment, I copy the whole segment to the file during initialization. This may be risky if another thread/entity can somehow update the data segment before the mmap succeeds. But I haven't seen any problems in practice. (MacOS does not support this because writing to the file fails for some reason.)
Thanks to @halitdogan and Pardo for the help on this!
@markbrown314 - do you think it's possible to use tmpfs instead of the shm_ APIs to avoid some of the tricky issues mentioned above?