OTLP HTTP Exporter and Loki Endpoint - Unexpected /v1/logs Path

[eunhwa-park]

Hello Loki team,

I am currently integrating OpenTelemetry with Loki for log collection using the otlphttp exporter. While configuring the exporter with the following endpoint:

    exporters:
      otlphttp:
        endpoint: "http://loki-loki-distributed-gateway.logging.svc.cluster.local/loki/api/v1/push"

I encountered an issue where the exporter appends /v1/logs to the configured endpoint, resulting in the following request being made:

POST /loki/api/v1/push/v1/logs This causes a failure since Loki expects logs to be pushed to the /loki/api/v1/push endpoint without any additional path.

[ opentelemetry logs ] 2024-09-04T02:11:14.820Z error exporterhelper/queued_retry.go:394 Exporting failed. The error is not retryable. Dropping data. {"kind": "exporter", "data_type": "logs", "name": "otlphttp", "error": "Permanent error: error exporting items, request to http://loki-loki-distributed-gateway.logging.svc.cluster.local/loki/api/v1/push/v1/logs responded with HTTP Status Code 404", "dropped_items": 9}go.opentelemetry.io/collector/exporter/exporterhelper.(*retrySender).send go.opentelemetry.io/[email protected]/exporter/exporterhelper/queued_retry.go:394go.opentelemetry.io/collector/exporter/exporterhelper.(*logsExporterWithObservability).send go.opentelemetry.io/[email protected]/exporter/exporterhelper/logs.go:135go.opentelemetry.io/collector/exporter/exporterhelper.(*queuedRetrySender).start.func1 go.opentelemetry.io/[email protected]/exporter/exporterhelper/queued_retry.go:205go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*boundedMemoryQueue).StartConsumers.func1 go.opentelemetry.io/[email protected]/exporter/exporterhelper/internal/bounded_memory_queue.go:61

The error message from the Nginx proxy in front of Loki is:

404 "POST /Loki/api/vl/push/vl/Logs HTTP/1.1" 19 HTTP/1.1" "Openlelemetry Collector Contr1b/0.70.0 (Linux/amd64)" "_"

Could you provide guidance on how to handle this behavior with Loki? Is there any way to configure Loki to work with the /v1/logs path?

Thank you, Joy

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like A clear and concise description of what you want to happen.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.