tempo icon indicating copy to clipboard operation
tempo copied to clipboard

Allow querying traces by span instrumentation "Library Name"

Open atsu85 opened this issue 11 months ago • 2 comments

Is your feature request related to a problem? Please describe. I'm trying to find traces with spans created by specific library name.

Describe the solution you'd like TraceQL query should support finding traces that have spans with specific library name. For example, smth like following pseudocode:

{ resource.service.name="skeleton-nestjs" && instrumentationLibrarySpans.instrumentationLibrary.name="@opentelemetry/instrumentation-http"}

to match traces similar to the example screenshot from Grafana: image

When exporting the example trace to JSON (from Grafana), then it looks like following:

{
  "batches": [
    {
      "resource": {
        "attributes": [
          ...
        ],
        "droppedAttributesCount": 0
      },
      "instrumentationLibrarySpans": [
        {
          "spans": [
            {
              "traceId": "1245cf2e87b6be30ed3700f1c126b557",
              "spanId": "0882ee038dfdb585",
              "parentSpanId": "0000000000000000",
              "traceState": "",
              "name": "GET /ping",
              "kind": "SPAN_KIND_SERVER",
              "startTimeUnixNano": 1710846323303000000,
              "endTimeUnixNano": 1710846323303740000,
              "attributes": [
                ...
              ],
              "droppedAttributesCount": 0,
              "droppedEventsCount": 0,
              "droppedLinksCount": 0,
              "status": {
                "code": 0,
                "message": ""
              }
            }
          ],
          "instrumentationLibrary": {
            "name": "@opentelemetry/instrumentation-http", // this is the attribute to match using TraceQL
            "version": "0.49.1"
          }
        }
      ]
    },
  ]
}

Additional context If it would be possible to find traces based on instrumentation library name, i could figure out the traces that are used for specific service (without looking at its code) to understand what instrumentation libraries could be disabled (to speed up application startup time that is super-slow when using auto-instrumentation for nodejs applications)

atsu85 avatar Mar 19 '24 11:03 atsu85

We have defined this in the language:

https://github.com/grafana/tempo/blob/main/docs/design-proposals/2023-11%20TraceQL%20Extensions.md#instrumentation-scope

but have yet to implement it. Currently all of our internal efforts are focused on delivering adhoc metrics from traces, but we do intend to add this eventually.

We would also accept an external contribution for this, but it might be difficult. We have had external TraceQL additions before but they require quite a deep dive.

Here's an example: https://github.com/grafana/tempo/pull/2848/files

joe-elliott avatar Mar 19 '24 12:03 joe-elliott

This issue has been automatically marked as stale because it has not had any activity in the past 60 days. The next time this stale check runs, the stale label will be removed if there is new activity. The issue will be closed after 15 days if there is no new activity. Please apply keepalive label to exempt this Issue.

github-actions[bot] avatar May 19 '24 00:05 github-actions[bot]

This issue has been automatically marked as stale because it has not had any activity in the past 60 days. The next time this stale check runs, the stale label will be removed if there is new activity. The issue will be closed after 15 days if there is no new activity. Please apply keepalive label to exempt this Issue.

github-actions[bot] avatar Jul 20 '24 00:07 github-actions[bot]

Based on my reading of https://github.com/grafana/tempo/blob/7961fb746daeea8f274db3e8ebbcac78b74f05b8/docs/design-proposals/2023-11%20TraceQL%20Extensions.md#instrumentation-scope, https://github.com/grafana/tempo/pull/3967 implements this.

devurandom avatar Aug 20 '24 11:08 devurandom

the PR #3967 that implements this feature is now merged :+1: (although the task was closed as stale before it got merged)

atsu85 avatar Aug 23 '24 07:08 atsu85