[tempo-distributed] Add OTel gRPC endpoints to nginx gateway

[AlexDCraig]

I'd like to hit the OTLP gRPC endpoints the distributor has. Looks to me like the chart only presently supports the http endpoints for nginx routing.

[agermain]

+1. Sounds like a duplicate of #733.

[ahaw023]

@AlexDHoffer

It is a little daft

I am using an OpenTelemetry collector and this is the config I set up to make it work

    exporters:
       otlphttp/grafana-hosted:
        endpoint: http://tempo-distributed-gateway.grafana-tempo.svc.cluster.local:80/otlp

[ahaw023]

longer term fix will be to add a listener for grpc and proxy back to the config

[AlexDCraig]

I don't know about others but I would be perfectly satisfied with circumventing the Tempo gateway pods entirely (i.e. let them just focus on routing http traffic) and simply adding an ingress resource that supports grpc traffic that routes to the distributor. I guess the question with this issue is how would you like to see distributor grpc endpoints exposed.

If what I said above is satisfactory to people I could write it and submit it, I just am pretty uncertain that makes everybody happy.

[artyom-p]

Also struggling with the same issue atm, tried to implement a workaround, the same as mentioned above - otlphttp exporter, but for some reason otlphttp cant send http headers for me, maybe im doing something wrong...

[artyom-p]

Ah, heres why its not working... https://github.com/grafana/tempo/issues/495

[sambacha]

This has been fixed in https://github.com/grafana/tempo/issues/495