gunzip failure when gRPC decompression is enabled
Stirling now supports gRPC decompression. However, during a brief testing on gke:prod cluster, gunzip failures were identified through the relevant log lines in the code.
One possible culprit is that the events are delivered to userspace out of order, and http2 tracing does not use DataStreamBuffer (as is used by kprobe-based pipeline). Those data will fail gunzip.
A corollary to the above speculation is that, prior to supporting grpc decompression, these out-of-order data events, them being serialized protobufs, would went through textformat protobuf parsing without explicit failure, because protobuf parsing tolerates such corruption in the input data.
We need to verify that data events are indeed received out-of-order, and if so, use DataStreamBuffer to store the data events.
Or find the true root cause(s).