tempo
tempo copied to clipboard
Why can not see traceID from loki?
Hello, I have set up k8s, and inside k8s, The promtail+loki+object sotrage works well. But now I want to see which part cost most time? Then I want to set up the grafana tempo. At first, I see that the config file of loki should be added these environment parameters in the statefulset.yaml : ----------- environment parameters begin ------------------- env: - name: JAEGER_AGENT_HOST value: tempo - name: JAEGER_ENDPOINT value: http://tempo:14268/api/traces - name: JAEGER_SAMPLER_TYPE value: const - name: JAEGER_SAMPLER_PARAM value: "1" ----------- environment parameters end ------------------- After this modification, I run this command to make it in use: kubectl apply -f statefulset.yaml -n loki
And then I query with grafana. However, I can not find any keyword about "traceID" from the query result. Is there anything wrong? Thanks very much.
The first thing we should do is simply confirm that Tempo is receiving traces. Two quick ways are to:
- Check
tempo_distributor_spans_received_total
andtempo_ingester_traces_created_total
to make sure these values are increasing. - The distributor has a config option "log_received_traces" for debugging that will simply print every traceid it receives. This is great for confirming that Tempo is receiving traces: https://grafana.com/docs/tempo/latest/configuration/#distributor
distributor:
log_received_traces: true
Thanks very much for your reply. In fact, I want to make clear that if without the tempo, only add the enviroment parameters to the Loki ,there will no data for traceID . Is that right? Thanks very much.
Loki is a very powerful way to find trace ids, but Tempo supports search natively as well. These links may help:
https://grafana.com/docs/tempo/latest/operations/backend_search/ https://grafana.com/docs/tempo/latest/getting-started/tempo-in-grafana/#tempo-search
This issue has been automatically marked as stale because it has not had any activity in the past 60 days. The next time this stale check runs, the stale label will be removed if there is new activity. The issue will be closed after 15 days if there is no new activity. Please apply keepalive label to exempt this Issue.