brpc icon indicating copy to clipboard operation
brpc copied to clipboard

brpc客户端多并发下卡在callmethod函数

Open woailunhua opened this issue 1 month ago • 4 comments

Describe the bug brpc客户端多并发下卡在callmethod函数,不会报错也不返回结果(一直卡死)我这里设置的是同步接口,服务端也收不到这十个请求,下面是gdb 堆栈信息: #0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38 #1 0x00007f1baf0261fe in bthread::futex_wait_private (timeout=0x0, expected=0, addr1=0x7f1b50ff2140) at ./src/bthread/sys_futex.h:40 #2 bthread::wait_pthread (pw=..., ptimeout=ptimeout@entry=0x0) at src/bthread/butex.cpp:142 #3 0x00007f1baf026950 in bthread::butex_wait_from_pthread (abstime=0x0, expected_value=1, b=0x7f1b24054210, g=) at src/bthread/butex.cpp:615 #4 bthread::butex_wait (arg=arg@entry=0x7f1b24054210, expected_value=expected_value@entry=1, abstime=abstime@entry=0x0) at src/bthread/butex.cpp:648 #5 0x00007f1baf02fb19 in bthread_id_join (id=...) at src/bthread/id.cpp:529 #6 0x00007f1baf0d82e9 in brpc::Channel::CallMethod (this=0x7f1b680d9b20, method=0x0, controller_base=0x7f1b50ff22f0, request=0x0, response=0x0, done=0x0) at src/brpc/channel.cpp:581

To Reproduce

Expected behavior 希望不要一直卡住,要么返回错误日志,要么返回结果。

Versions OS:linux Ubuntu 22.04.4 LTS Compiler: c++17 brpc: 项目经过多人,不清楚是哪个版本的brpc protobuf:基于http协议,没有用到

Additional context/screenshots

woailunhua avatar Nov 27 '25 02:11 woailunhua

查看controller详细状态 明显能看到错误码,以及连接超时这些状态,为什么这个情况下,callmethod不会返回错误状态呢? (gdb) p (brpc::Controller)controller_base $3 = {google::protobuf::RpcController = {<No data fields>}, static FLAGS_IGNORE_EOVERCROWDED = 1, static FLAGS_SECURITY_MODE = 2, static FLAGS_ADDED_CONCURRENCY = 4, static FLAGS_READ_PROGRESSIVELY = 8, static FLAGS_PROGRESSIVE_READER = 16, static FLAGS_BACKUP_REQUEST = 32, static FLAGS_DESTROY_CID_IN_DONE = 128, static FLAGS_CLOSE_CONNECTION = 256, static FLAGS_LOG_ID = 512, static FLAGS_REQUEST_CODE = 1024, static FLAGS_PB_BYTES_TO_BASE64 = 2048, static FLAGS_ALLOW_DONE_TO_RUN_IN_PLACE = 4096, static FLAGS_USED_BY_RPC = 8192, static FLAGS_PB_JSONIFY_EMPTY_ARRAY = 65536, static FLAGS_ENABLED_CIRCUIT_BREAKER = 131072, static FLAGS_ALWAYS_PRINT_PRIMITIVE_FIELDS = 262144, static FLAGS_HEALTH_CHECK_CALL = 524288, static FLAGS_PB_SINGLE_REPEATED_TO_ARRAY = 1048576, static FLAGS_MANAGE_HTTP_BODY_ON_ERROR = 2097152, _span = 0x0, _flags = 10240, _error_code = 1008, _error_text = "[E110]Fail to connect Socket{id=791 addr=10.121.34.12:32829} (0x0x7f47ec0c4e60): Connection timed out [R1][E1008]Reached timeout=500ms @10.121.34.12:32829", _remote_side = {ip = { s_addr = 203585802}, port = 32829}, _local_side = {ip = {s_addr = 0}, port = 0}, _session_local_data = 0x0, _server = 0x0, _oncancel_id = {value = 0}, _auth_context = 0x0, _mongo_session_data = { px = 0x0}, _sampled_request = 0x0, _request_protocol = brpc::PROTOCOL_HTTP, _max_retry = 3, _retry_policy = 0x0, _correlation_id = {value = 7696581394433}, _connection_type = brpc::CONNECTION_TYPE_POOLED, _fail_limit = -123456789, _pipelined_count = 0, _timeout_ms = 500, _connect_timeout_ms = 400, _backup_request_ms = -1, _real_timeout_ms = 100, _deadline_us = 1764229948986363, _timeout_id = 8589936640, _begin_time_us = 1764229948486363, _end_time_us = 1764229948486363, _tos = 0, _preferred_index = 7, _request_compress_type = brpc::COMPRESS_TYPE_NONE, _response_compress_type = brpc::COMPRESS_TYPE_NONE, _inheritable = {log_id = 0, request_id = ""}, _pchan_sub_count = 0, _response = 0x0, _done = 0x0, _sender = 0x0, _request_code = 0, _single_server_id = 0, _lb = {px = 0x0}, _tmp_completion_info = {id = { value = 7696581394433}, responded = false}, _current_call = {nretry = 1, need_feedback = false, enable_circuit_breaker = false, touched_by_stream_creator = false, peer_id = 0, begin_time_us = 1764229948886448, sending_sock = std::unique_ptrbrpc::Socket = {get() = 0x7f48040c6b40}, stream_user_data = 0x0}, _unfinished_call = 0x0, _accessed = 0x0, _stream_creator = 0x0,

_pack_request = 0x7f488a5e5340 <brpc::policy::PackHttpRequest(butil::IOBuf*, brpc::SocketMessage**, unsigned long, google::protobuf::MethodDescriptor const*, brpc::Controller*, butil::IOBuf const&, brpc::Authenticator const*)>, 

woailunhua avatar Nov 27 '25 08:11 woailunhua

主动设置一下 timeout 时间(比如 1s),他就会超时返回,否则它会等到默认的超时时间结束才会返回

royguo avatar Dec 09 '25 03:12 royguo

设置了的 没有用。 你可以看到timed out 500ms。并且它达到超时时间了也不会退出

---原始邮件--- 发件人: "Roy @.> 发送时间: 2025年12月9日(周二) 中午11:04 收件人: @.>; 抄送: @.@.>; 主题: Re: [apache/brpc] brpc客户端多并发下卡在callmethod函数 (Issue #3164)

royguo left a comment (apache/brpc#3164)

主动设置一下 timeout 时间(比如 1s),他就会超时返回,否则它会等到默认的超时时间结束才会返回

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

woailunhua avatar Dec 09 '25 03:12 woailunhua

设置了的 没有用。 你可以看到timed out 500ms。并且它达到超时时间了也不会退出

我跟你的使用场景完全一样,我观察到可以正常 timeout 退出

royguo avatar Dec 09 '25 04:12 royguo