parsec
parsec copied to clipboard
Parsec context shared between multiple libraries
There is a need to be able to share the same parsec context between multiple libraries. Typically MADNESS (over parsec) and TTG (also over parsec) will need to co-exist in some applications.
In the current approach, one library (MADNESS in this case) initiates the parsec context and exposes it via a function in its API, and the other library (TTG in this case) uses this same context to support its execution.
Separation is guaranteed by the use of different taskpools for both libraries.
In the current implementation, the use of parsec is hidden by both libraries: the user-facing API of both libraries does not expose a parsec context. This makes sense for both libraries, because both are ported over multiple runtime systems, and ideally the application developer should not need to be aware of which runtime system is used for a given application.
This creates two issues:
- library A does not know if library B has been initialized before or not. Said otherwise, we don't have an
MPI_Initialized()
function, or aparsec_get_default_context()
one. Is this something that we could/would provide? Or how do we propose different libraries to solve this issue? - library A has initialized the parsec context and has found a way to share it with library B. But library A has not initialized it like library B wants it (typically, MADNESS would initialize parsec with MPI_COMM_SELF by default has it doesn't use parsec to communicate, but TTG needs to change that comm to a dup of MPI_COMM_WORLD). The problem is that library A has a taskpool already registered in the shared parsec context, so changing the context is not an option... I don't see a workaround this one: it looks to me that either A and B need to negotiate beforehand which communicator to use for both, or A and B should have a function to remove its taskpool from the context, and re-create it / re-add it to the context.... Any better idea?