openucx
Move Common UCS Code from UCX into UCC
What
Moves the UCS code that's used by all TLs from the UCX repository into the UCC repository. At the end of the conversion, when not compiling with something that requires UCX otherwise (e.g., UCP TL), UCC will not link against libucx.so/libucs.so/libucm.so.
Why ?
See the thread for all of the background, but the top reasons are:
- The UCS/UCM code was not written with the intention of being used outside of the UCX library. The API is not necessarily stable or backward/forward compatible. UCC has been bitten in the past by changes inside UCX that caused breakages on the UCC side.
- Meta doesn't want production teams to have to maintain (or be familiar with) libraries that we're not using. Currently, we have to bring in knowledge of UCX in order to build UCC (even if we turn off the UCP TL). Obviously, if we're using the UCP TL, we need UCX, but that's not expected to be the common case.
- Even if we just use the UCS/UCM code inside UCX, we still have to know the UCX codebase well enough to know what parts are relevant and what parts are not.
How ?
This PR will bring over the code that is used by both UCC (outside of TLs that require UCX anyway) and UCX itself. Primarily, this means that everything aside from rcache will be brought over from UCX to UCC.
Most of the code will go into the src/utils directory, but there is a little bleeding outside of that directory when necessary.
Obviously this PR is still very much WIP, but I wanted to let folks get an idea of the direction I was thinking here. It'll probably take me a while to get everything moved over.
