received message after decompression larger than max (4331797 vs. 4194304)

[zhulei-pacvue]

Component(s)

No response

Describe the issue you're reporting

During the operation of otelcol, errors are often reported, and the specific error content is as follows: 2024-04-22T07:17:12.488Z error exporterhelper/queue_sender.go:101 Exporting failed. Dropping data. {"kind": "exporter", "data_type": "traces", "nameermanent error: rpc error: code = ResourceExhausted desc = grpc: received message after decompression larger than max (4430579 vs. 4194304)", "dropped_items": 441} go.opentelemetry.io/collector/exporter/exporterhelper.newQueueSender.func1 go.opentelemetry.io/collector/[email protected]/exporterhelper/queue_s

Otelcol version： 0.97.0

CPU usage and memory usage：

Main configurations：

connectors:
  spanmetrics:
    dimensions:
      - name: http.method
      - name: http.status_code
      - name: http.route
    exemplars:
      enabled: false
    events:
      enabled: false
      dimensions:
        - name: exception.type
        - name: exception.message
exporters:
  otlp:
    endpoint: 'jaeger-collector:4317'
    tls:
      insecure: true 
    compression: "gzip"
    timeout: 1s
    sending_queue:
      enabled: true
      num_consumers: 10
      queue_size: 10000
  prometheus:
    endpoint: ${env:MY_POD_IP}:8889
    namespace:
    send_timestamps: false
    metric_expiration: 10m
    add_metric_suffixes: false   
extensions:
  zpages:
    endpoint: ${env:MY_POD_IP}:55679
  health_check:
    endpoint: ${env:MY_POD_IP}:13133
  pprof:
    endpoint: ${env:MY_POD_IP}:1777
processors:
  batch:
    timeout: 1s
    send_batch_size : 100
  filter/ottl:
    error_mode: ignore
    metrics:
      metric:
      - name == "rpc.server.duration"
      - name == "duration"

Meanwhile, how to modify the limitations of GRPC ` s 4M

[crobert-1]

From a quick search, it looks like a GRPC setting would need to be updated in the exporter helper code to handle the size of message that you're attempting to send.