Remote dependency requirements for TTG

[devreal]

Description

Because I am tired of repeating why we cannot use PaRSEC's remote dependencies in TTG and instead use the parsec_ce directly, here is a list of what I think is needed.

• [ ] Handling of arbitrary, nested objects with iovecs describing the nested data.*
• [ ] Packing of parts of the object (data that is not worth expressing as an iovec) and reconstruction from that data.
• [ ] Allocation and construction of objects before transfer into the provided iovecs.*
• [ ] Handling of arbitrary task identifiers and mapping functions of task identifiers to processes and transfer of arbitrary-length list of task identifiers between processes. The model of TTG allows for tasks to describe their successors as list of identifiers, which cannot be collapsed into a simpler (generative) expression in all cases. These identifier lists are the only way receiving processes know what successors receive a data as there is now global knowledge about the task graph (like in PTG or DTD).

• Essentially PaRSEC would need a version of https://conferences.computer.org/sc-wpub/pdfs/SC-W2024-6oZmigAQfgJ1GhPL0yE3pS/555400a414/555400a414.pdf

I don't care whether one or the other feature is available in some stale branch. As long as no one brings it into mainline PaRSEC it might as well not exist.

This is a start. I will add more things when I come across them.

[omor1]

The PaRSEC remote dependency model inherently relies on an assumption that the destination can reconstruct all (or at least the local) successors of a predecessor task. This enables the runtime to skip sending the (variable-length) list of successors at the cost of some extra computation at the destinations.

TTG intentionally relaxes this assumption so that it can provide e.g. data-dependent task graphs—as you said, there is "no global knowledge about the task graph (like in PTG or DTD)". Without massively changing the PaRSEC remote dependency model, it is simply incompatible with TTG.