Looking for OpenTelemetry go tracing example with spans propagating through the gateway

[kye308]

🚀 Feature

Are there any examples of something like this using OpenTelemetry's go SDK? Would a tracing wrapper still be required?

[johanbrandhorst]

I don't know of any examples of using the gateway with the Otel Go SDK. You would still need a tracing wrapper yes, it should be very similar to OpenTracing. Would you like to contribute an opentelemetry example?

[matoous]

I think this should be re-open, as mentioned on the issue in the examlple that was added in the PR that closed this issue, the propagation doesn't work.

[johanbrandhorst]

Sorry to hear it's not working. Would you be interested in submitting a fix?

[johanbrandhorst]

CC @iamrajiv

[iamrajiv]

Hi @johanbrandhorst, I'm currently investigating the issue and working on a solution.

[matoous]

@iamrajiv here's a working fix: https://github.com/openfga/openfga/pull/1024/files, I just didn't have time to update this issue yet. One needs to wrap the runtime.ServerMux in otelhttp.NewHandler (or alternatively in the PR above we just extract the trace context and propagate it on context.Context) and the same needs to be done for outgoing calls, either by wrapping the gRPC client in UnaryClientInterceptor/StreamClientInterceptor (https://pkg.go.dev/go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc#UnaryClientInterceptor) or manually to skip the span:

			propagator := otel.GetTextMapPropagator()
			// If tracig is enabled wrap the handler with custom middleware that extracts
			// the tracing context and sets the corrent tracing context.
			// We use this in favor of [otelhttp.NewHandler] to avoid unnecessary nesting.
			handler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
				ctx := propagator.Extract(r.Context(), propagation.HeaderCarrier(r.Header))
				handler.ServeHTTP(w, r.WithContext(ctx))
			})

If using the otel package for Golang this results into 2 additional spans so the PR above does this manually to skip both of them and make the grpc-gateway transparent in the trace call stack.