osc/ucx does not honor thread level requested

[edgargabriel]

The osc/ucx component does not honor the thread level requested when calling MPI_Init_thread. The main culprit is the line here:

https://github.com/open-mpi/ompi/blob/main/opal/mca/common/ucx/common_ucx_wpool.c#L58

which initializes the worker used by the osc component to be always UCS_THREAD_MODE_SINGLE. This issue didn't surface previously because prior to ucx 1.18.0 the check for which thread is executing an operation was incomplete. However, starting from this commit in UCX

https://github.com/openucx/ucx/commit/bdd207e6ba5c9dc3cf2acfd54560380456f634e3

we are now getting errors of

ucp_worker.c:3055 Assertion `ucs_async_check_owner_thread(&(worker)->async)

when executing MPI_Put/Get calls in a thread other than the main thread, despite of having initialized MPI with THREAD_MULTIPLE support. A simple workaround is to configure UCX with --disable-assertions, but this is still an error in my opinion.