ompi
ompi copied to clipboard
open-mpi
btl: ofi add RMA/ATOMIC caps for one-sided to work
MCA_BTL_OFI_ONE_SIDED_REQUIRED_CAPS is only set in the hints.caps
cxi provider checks the tx_attr.caps for these capabilities explicitly. Therefore, if not set then the operation fails.
One update to the patch is to use the define instead of explicitly specifying the caps
It could be that I'm running with older CXI code. With the version of the code I run with. The context is allocated with the caps passed in from MPI fi_tx_context() call. That's 0. Therefore the internal capabilities stored against the cxi transmit context is 0. When fi_read() is called it checks that RMA/ATOMIC capabilities are set and because they are not the fi_read() fails.
I'm looking at the code that they have in a PR in libfabric repo and it's very different than what I have.
How did you verify that the CXI provider works with one-sided? IE which code did you use?
ok. i brought the issue that I saw to the attention of the CXI developers. We'll see what they say.
can we close this? This is trying to work around problems in pre HPE SS11 2.2.x releases of CXI provider.
If we are sure it's something which has already been fixed in cxi provider then I'm good with closing it. I haven't heard back from them about it.