scaproust icon indicating copy to clipboard operation
scaproust copied to clipboard

Intermittent recv_when_inactive error with large REQ/REP messages

Open robsmith11 opened this issue 6 years ago • 1 comments

Unfortunately, I don't have a reliable way to reproduce this, but it seems to happen more often (maybe 25% of the time) with large messages (~10MB). I'm using IPC on Linux. Any idea what's wrong or how to debug?

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Custom { kind: Other, error: StringError("Can\'t recv: no active request")\
 }', libcore/result.rs:945:5

robsmith11 avatar May 08 '18 10:05 robsmith11

This error is returned by recv when the REQ socket is not expecting a reply, ie when there is no matching successful send. You can get this error when trying the receive a reply without sending a request first, or when sending the request has failed. To debug this you could enable the logs and watch for the protocol state transitions. If you use env_logger, this is how you activate the scaproust logs. If you don't use env_logger, please have a look at the provided reqrep example to see how to use it.

export RUST_LOG=scaproust

Then you can redirect the stderr of your process and grep for the req state transitions.

./target/debug/examples/reqrep node1 ipc://reqrep 2>&1 | grep 'scaproust::proto::req'

Below is the output given by the reqrep example.

DEBUG:scaproust::proto::req: [Socket:0] switch from Idle to SendOnHold
DEBUG:scaproust::proto::req: [Socket:0] switch from SendOnHold to Sending
DEBUG:scaproust::proto::req: [Socket:0] switch from Sending to Active
DEBUG:scaproust::proto::req: [Socket:0] switch from Active to Receiving
DEBUG:scaproust::proto::req: [Socket:0] switch from Receiving to Receiving
DEBUG:scaproust::proto::req: [Socket:0] switch from Receiving to Receiving
DEBUG:scaproust::proto::req: [Socket:0] switch from Receiving to Idle

The line Sending to Active occurs when the send is successful, otherwise you would see a Sending to Idle transition. Scaproust will write quite a lot of logs so it will help if there is only one socket in the process.

blabaere avatar May 08 '18 14:05 blabaere