promscale
promscale copied to clipboard
Add telemetry event for Jaeger gRPC remote storage writer
Relates to #1535
@VineethReddy02 We already have telemetry event around otel ingestion and #1535 is implemented on top of otel. Hence all the events which we send for otel ingestion is also applicable for Jaeger ingestion. Do we need any additional signal from the product perspective?
I agree with your point Arun, But we need to know if the user is using Otel or Jaeger ingestion endpoints or both? And rest all we can see from Otel-specific telemetry that has already been captured. Maybe it's a good idea to change telemetry keys from otel
to traces
to make it more generic. So that in telemetry the data points with traces
represents traces in general i.e both Jaeger and Otel.
I've been exploring following options to enable telemetry,
- Add metrics in WriteSpan and expose it to telemetry ( similar to e.g. https://github.com/timescale/promscale/blob/master/pkg/jaeger/store/store.go#L64)
- Expose intrinsic metrics exposed by gRPC framework(grpc_server_handling_seconds_count{grpc_method="WriteSpan"})
- Use internal span attribute populated by jaeger framework(internal.span.format=jaeger)
Option 1 and Option 2 requires code changes in promscale and extension, but 3 requires change only in extension.