Flag UCP_OP_ATTR_FLAG_FORCE_IMM_CMPL is ignored by ucp_put_nb when UCX_PROTO_ENABLE=y

[ztiffany]

Describe the bug

Flag UCP_OP_ATTR_FLAG_FORCE_IMM_CMPL is ignored by ucp_put_nb when UCX_PROTO_ENABLE=y

Steps to Reproduce

This is reproduced with a private test. I can provide it if necessary. ucp_put_nb returns a request in some situations when FORCE_IMM_CMPL and PROTO_ENABLE are true. I expect this to be invalid and my test fails.

When I set PROTO_ENABLE=n, my test passes. When I remove the use of the FORCE_IMM_CMPL flag, my test passes.

Setup and versions

Using master from 2/14 (1acd1f2d).

Additional information (depending on the issue)

[tvegas1]

Would patch below help? @yosefe, @brminich, is the intent correct for proto enable yes?

diff --git a/src/ucp/rma/rma_send.c b/src/ucp/rma/rma_send.c
index 2e6d659..13ec60c 100644
--- a/src/ucp/rma/rma_send.c
+++ b/src/ucp/rma/rma_send.c
@@ -271,6 +271,11 @@ ucs_status_ptr_t ucp_put_nbx(ucp_ep_h ep, const void *buffer, size_t count,
             goto out_unlock;
         }

+        if (ucs_unlikely(param->op_attr_mask & UCP_OP_ATTR_FLAG_FORCE_IMM_CMPL)) {
+            ret = UCS_STATUS_PTR(UCS_ERR_NO_RESOURCE);
+            goto out_unlock;
+        }
+
         req = ucp_request_get_param(worker, param,
                                     {ret = UCS_STATUS_PTR(UCS_ERR_NO_MEMORY);
                                     goto out_unlock;});

[brminich]

patch LGTM with minor comment that maybe better to do else if to the previous branch