opentelemetry-erlang-contrib
opentelemetry-erlang-contrib copied to clipboard
No way to set max compression size limit
Describe the bug Maybe not really a bug but more a lack of configuration or adherence to the spec.
We're seeing this kind of error:
OTLP grpc export failed with GRPC status 8 : grpc: received message after decompression larger than max (13104533 vs. 4194304)
Which looks like the message is too large and hits the 4194304 size limit. However, some people like in this issue here seem to have configurable ways of increasing this limit.
Possibly a workaround is to play with these env vars decreasing their scheduled delay and max queue size to deliver messages quicker before they reach this size limit?
Expected behavior A way to avoid this error, either by a new configuration or some documentation explaining how to tackle it.
Additional context
@sashaafm you are seeing this error in tempo?
No, this happens in the Elixir console. We’re using Datadog as a metrics aggregator/visualizer. However, this happens in the instance where the app is running (alongside the Datadog Agent).
Adding here what I said in Slack so it isn't lost in time :)
We don't yet support OTEL_BSP_MAX_EXPORT_BATCH_SIZE but do have OTEL_BSP_SCHEDULE_DELAY_MILLIS (bsp_scheduled_delay_ms in the application configuration) which defaults to 5 seconds.