Random crashing when sending large amounts of data through a stream (openssl)

[Ten0]

Describe the bug Random crashing when sending large amounts of data through a stream.

We get the error: RpcStatus { status: Unknown, details: Some("Stream removed") } The following lines are logged by grpc just before the error is returned:

1 ssl_transport_security.cc:472] Corruption detected.
1 ssl_transport_security.cc:448] error:0906D06C:PEM routines:PEM_read_bio:no start line
1 secure_endpoint.cc:208]     Decryption error: TSI_DATA_CORRUPTED

To Reproduce Unfortunately, the crashing is random, so I don't have a magic recipe to create it. The idea is i'm sending large amounts of data through a bi-dir stream, which are then buffered 500 by 500 on the server side then answers are sent back to the client.

Expected behavior Stream to process normally.

System information

• CPU architecture: x64
• Distribution and kernel version: docker debian:stretch
• Any other system details we should know?: Using the openssl version (stretch's openssl package with feature openssl)

I need to link against openssl because I'm using other crates that depend on specifically openssl (and not google's fork).

[BusyJay]

Thanks for the report. Are both client and server using the grpcio library?

[Ten0]

Yes.

[BusyJay]

Which version are you using?

[Ten0]

I'm using something somewhat close to master: fec97d19 with gRPC Core updates & openssl build fail patches backported (precisely, Ten0/grpc-rs@d093dc31).

Server and client are not on the same machine. The OS is debian stretch in a Docker.