ompi icon indicating copy to clipboard operation
ompi copied to clipboard

MPI 4.1 - implement memory allocation kind info

Open hppritcha opened this issue 1 year ago • 5 comments
trafficstars

This is a biggy and likely important to customers using Open MPI with GPU accelerators.

See https://github.com/mpi-forum/mpi-issues/issues/580 and entry 17 of section B.1.2 of the MPI 4.1 standard

hppritcha avatar Dec 21 '23 18:12 hppritcha

Details:

  • section 11.4.3 (Memory Allocation Info) of MPI 4.1 contains the description of the MPI memory allocation kinds concept
  • section 11.5 (Portable MPI Process Startup) contains the new argument to pass memory allocation kind to mpiexec
  • Side document (https://github.com/mpi-forum/mem-alloc) contains the name of the info values for the various accelerators (cuda, rocm, ze)

edgargabriel avatar Mar 28 '24 14:03 edgargabriel

Just an FYI: you'll need to add an attribute definition to the PMIx standard if you want this to work with OMPI v5 and above - putting something in the MPIR side doc won't do anything for you, I'm afraid. I will add a PMIX_MEM_ALLOC_KIND definition to the PMIx library just so the support exists. The debugger guys look at the PMIx Standards doc, so somebody needs to get that added to it.

rhc54 avatar Mar 28 '24 15:03 rhc54

Also, could you perhaps add the list of the info values you mention (in case we need attributes for those too)? I can't leaf thru that side doc repo to find them, and they aren't shown anywhere obvious.

rhc54 avatar Mar 28 '24 15:03 rhc54

related to https://github.com/openpmix/prrte/pull/1972

hppritcha avatar May 02 '24 15:05 hppritcha

I have added the PMIx support and support for PMIx-based tools in PRRTE: see https://github.com/openpmix/prrte/pull/1973 and https://github.com/openpmix/openpmix/pull/3342. Still not sure if we need to worry about defining attributes for your identified values - is there such a need?

rhc54 avatar May 02 '24 21:05 rhc54