otlphttp exporter is failing

[jangaraj]

I configured otlphttp exporter:

exporters:
  otlphttp:
    endpoint: https://otel-collector.domain.com

But it is failing sometimes, example errors:

{"kind": "exporter", "name": "otlphttp", "error": "failed to make an HTTP request: Post \"https://otel-collector.domain.com/v1/traces\": net/http: TLS handshake timeout", "interval": "5.52330144s"}
{"kind": "exporter", "name": "otlphttp", "error": "failed to make an HTTP request: Post \"https://otel-collector.domain.com/v1/traces\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)", "interval": "4.688570936s"}

https://otel-collector.domain.com:443 is where my http OTLP receiver is listening (actually, it is ECS Fargate service behind ALB).

I tried to increase Lambda memory (128->512MB) to have more CPU allocation, less secure security policy on ALB, Lambda in my customer VPC/outside VPC, but iT didn't help. I can still see this error sometimes. it is a intermittent issue.

OTLP/GRPC endpoint (port 4317 in my case) works fine with otlp exporter. I'm also not able to replicate issue with protocurl command, so it looks like AWS infrastructure issue.

Any ideas how to improve/fix reliability for OTLP/HTTP protocol?

[bryan-aguilar]

Thank you for the contribution. As this is an intermittent issue do you have any steps where we could reliable setup a test environment to see this error? There seems to be quite a few specifics for your environment that would need to be replicated.

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. If you want to keep this issue open, please just leave a comment below and auto-close will be canceled

[github-actions[bot]]

This issue was closed because it has been marked as stale for 30 days with no activity.