hydra: refactor pmi message handling

[hzhou]

Pull Request Description

Goals of this PR:

• Making the code logic that deals with wire protocols and semantic protocols orthogonal
• Making the protocol logic (wire or semantic) orthogonal to functionality
• Use the better macros to make code more succinct

UPDATE:

• split cleanup commits into https://github.com/pmodels/mpich/pull/6104
• import pmi (just as mpl)
• use the wire routines from pmi/src/pmi_wire.h.

[skip warnings]

Author Checklist

• [x] Provide Description Particularly focus on why, not what. Reference background, issues, test failures, xfail entries, etc.
• [x] Commits Follow Good Practice Commits are self-contained and do not do two things at once. Commit message is of the form: module: short description Commit message explains what's in the commit.
• [ ] Passes All Tests Whitespace checker. Warnings test. Additional tests via comments.
• [x] Contribution Agreement For non-Argonne authors, check contribution agreement. If necessary, request an explicit comment from your companies PR approval manager.

[hzhou]

test:mpich/pmi

[hzhou]

6000 lines of diff over 25 commits. Please split this into smaller pieces.

Sure.

[hzhou]

test:mpich/pmi

[hzhou]

test:mpich/ch4/ofi test:mpich/pmi

[hzhou]

Is this the last in this series or is there more?

Yeah, this is the last of the previous big PR, but it is the base of new rounds of work :) Plan --

• factor out the message protocol. The current code dealing with key names, setting and checking "rc", "thrid" etc can be unified and handled by the utility layer
• https://github.com/pmodels/mpich/pull/5943
• Add binary keyval extension
• Add minimum PMIx in hydra and libpmi, just the set that MPICH currently uses.

[raffenet]

Is this the last in this series or is there more?

Yeah, this is the last of the previous big PR, but it is the base of new rounds of work :) Plan --

OK, I see.

• factor out the message protocol. The current code dealing with key names, setting and checking "rc", "thrid" etc can be unified and handled by the utility layer
• hydra: WIP toward reuse proxy for spawning and tree-launch #5943
• Add binary keyval extension
• Add minimum PMIx in hydra and libpmi, just the set that MPICH currently uses.

I thought we had agreed that I would implement the PMIx support. I hope that is still the case.

[raffenet]

One concern I have while browsing the new code is that proxy/server functions that support only PMI1 or PMI2 are now difficult to distinguish from functions that support both. I'm not sure this is actually desirable.