opentelemetry-go icon indicating copy to clipboard operation
opentelemetry-go copied to clipboard

Add handling for grpc MaxMessageSize on otlptracegrpc and otlpmetricgrpc exports

Open e96wic opened this issue 2 years ago • 2 comments

Problem Statement

When exporting spans using otlptracegrpc a golang server possibly responds with received message larger than max. The golang grpc max message size is 4MB.

Proposed Solution

In otlptracegrpc we could check the size of a grpc message before sending and since we have an array of protoSpans we could batch with that: https://github.com/open-telemetry/opentelemetry-go/blob/1b02c9122dadecd52e94dfbe3d456593da884e84/exporters/otlp/otlptrace/otlptracegrpc/client.go#L203

There are ways of checking the message size of a proto message: link

Alternatives

Using a batch exporter, we could limit the batch size of exported spans. Batch size and grpc message size are not necessarily related however, so the problem would not be fixed using this option. I assume adding a lot of data (attributes, ...) to the processed spans will increase the message size, so we would be playing catch-up with our batch sizes.

sdktrace.WithBatcher(exporter, sdktrace.WithMaxExportBatchSize(256))

Prior Art

A clear and concise list of any similar and existing solutions from other projects that provide context to possible solutions.

Additional Context

e96wic avatar Jun 27 '23 09:06 e96wic

I guess the same issue applies for otlpmetricgrpc

pellared avatar Jun 29 '23 07:06 pellared