PROTOCOL_H2 serialization not working with braft

[MayurSadavarte]

Describe the bug (描述bug) I am trying to run braft atomic application with brpc code while setting protocol as HTTP2 in channel options. Raft leader node encounters following error in HTTP2 serialization: W1030 22:23:53.358311 78030 replicator.cpp:393] Group Atomic fail to issue RPC to 10.48.221.246:8300:0 _consecutive_error_times=1, [E1003]request_attachment must be empty when request is not NULL

While looking at the code I found following problem: Http serialization method (SerializeHttpRequest) has logic that says that if client has provided non-null pbreq, Controller must not have non-empty attachment.

void SerializeHttpRequest(butil::IOBuf* /not used/, Controller* cntl, const google::protobuf::Message* pbreq) { .... if (pbreq != NULL) { // If request is not NULL, message body will be serialized proto/json, if (!cntl->request_attachment().empty()) { return cntl->SetFailed(EREQUEST, "request_attachment must be empty " "when request is not NULL"); } }

If we use brpc library with braft library, replicator code in braft (replicator.cpp), calls brpc channel method by providing both protocol buffer request and controller attachment:

void Replicator::_send_entries() { .... std::unique_ptr<AppendEntriesRequest> request(new AppendEntriesRequest); .... for (int i = 0; i < max_entries_size; ++i) { prepare_entry_rc = _prepare_entry(i, &em, &cntl->request_attachment()); .... } .... stub.append_entries(cntl.release(), request.release(), response.release(), done); .... }

Basically, Raft replicator sends log entries metadata within protobuf and log entries data serialized within controller attachment. Channel code is passing these two entities along to protocol serialization code as is:

void Channel::CallMethod(const google::protobuf::MethodDescriptor* method, google::protobuf::RpcController* controller_base, const google::protobuf::Message* request, google::protobuf::Message* response, google::protobuf::Closure* done) { ..... // Note that here request is non-null and also cntl has _serialize_request(&cntl->_request_buf, cntl, request); .... }

As a fix, we either need to fix Braft replicator to be channel protocol aware or fix Brpc channel code to restructure the request before calling protocol serialization code, or fix protocol serialization to handle a condition where both protocol request and attachment are set.

To Reproduce (复现方法)

1. Compile braft atomic application with brpc.
2. Set protocol as 'PROTOCOL_H2' in channel options.
3. Run raft servers and client.

Expected behavior (期望行为) Brpc when used with Braft with protocol set to HTTP, should work!

Versions (各种版本) OS: Centos 7 Compiler: Clang brpc: 0.9.7 protobuf: 3

Additional context/screenshots (更多上下文/截图)

[idilkara]

Hi, I am having the same issue when I am running the counter example of braft with brpc protocol set to PROTOCOL_H2.