OpenTelemtry and Sentry traces not connected

[peter-svensson]

Environment

SaaS (https://sentry.io/)

What are you trying to accomplish?

I'm trying to use the golang sentry sdk together with the otel sdk. My scenario is that I have a module dependency that is using otel to start a trace (transaction in Sentry terms?) when a message is received from (for example) rabbitmq. The code for this would be similar to this:

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/getsentry/sentry-go"
	"github.com/getsentry/sentry-go/otel"
	"go.opentelemetry.io/otel"
	sdktrace "go.opentelemetry.io/otel/sdk/trace"
	"go.opentelemetry.io/otel/trace"
)

func main() {
	opts := sentry.ClientOptions{
		Dsn:         "",
		Environment: "development",
	}
	if err := sentry.Init(opts); err != nil {
		log.Fatalf("sentry setup, %w", err)
	}

	tp := sdktrace.NewTracerProvider(
		sdktrace.WithSpanProcessor(
			sentryotel.NewSentrySpanProcessor()),
	)
	otel.SetTracerProvider(tp)
	otel.SetTextMapPropagator(sentryotel.NewSentryPropagator())

	// This could be in a dependency that starts a trace (for example a message queue receiving a message)
	tracer := otel.Tracer("dependency")
	tracingCtx, span := tracer.Start(context.Background(), "start",
		trace.WithSpanKind(trace.SpanKindConsumer),
	)
	fmt.Println("traceID in dependency", span.SpanContext().TraceID())
	defer span.End()

	// Passing the tracingCtx I would expect that the trace initiated would be used in the Sentry span here...
	sentrySpan := sentry.StartSpan(tracingCtx, "sentry")
	defer sentrySpan.Finish()
	fmt.Println("traceID in sentry", sentrySpan.TraceID)	
}

The problem that I'm facing is that since the TraceId is not the same in Sentry and Otel the traces will not be connected properly.

Running the code (with some more Sentry config)

opts := sentry.ClientOptions{
		Dsn:                "https://xxxxxxxxx",
		Environment:        "development",
		EnableTracing:      true,
		SampleRate:         1.0,
		ProfilesSampleRate: 1.0,
		Debug:              true,
		TracesSampleRate:   1.0,
	}

Gives the following output:

traceID in dependency dcebee1d8fa4c89e7b3fa4753f878b6f
traceID in sentry 8335e1afd6f236c4b11b8ac82aff2bf1
[Sentry] 2024/06/25 10:48:29 Sending transaction [9fb9fe33c534482e90be4f059567b344] to o4507442608668672.ingest.de.sentry.io project: xxx
[Sentry] 2024/06/25 10:48:29 Sending transaction [610065c082444b1a863f0104a062827d] to o4507442608668672.ingest.de.sentry.io project: xxx
[Sentry] 2024/06/25 10:48:29 Buffer flushed successfully.

So both spans are sent to Sentry, but since the root trace (transaction) created in otel is not the parent (transaction) for the Sentry span they are not associated..

This is no surprise if we check the code for StartPan:

func StartSpan(ctx context.Context, operation string, options ...SpanOption) *Span {
	parent, hasParent := ctx.Value(spanContextKey{}).(*Span)

and the otel sdk stores the span information:

const currentSpanKey traceContextKeyType = iota

// ContextWithSpan returns a copy of parent with span set as the current Span.
func ContextWithSpan(parent context.Context, span Span) context.Context {
	return context.WithValue(parent, currentSpanKey, span)
}

And since the SpanProcessor OnStart is not able to modify/return a new context it's not a surprise that the otal span is not found and considered the root (transaction).

How are you getting stuck?

Otel spans are not connected (as trace root/transaction) with sentry spans.

Where in the product are you?

Other

Link

No response

DSN

No response

Version

No response