Traces data is not visible in tempo db

[srinivasmummareddy]

Describe the bug I have setup Grafana Tempo on aks and I have exposed distributor service as loadbalancer and using azure blob storage . I have tried to send the trace data to tempo but I didn’t see any logs in distributor pod To Reproduce Steps to reproduce the behavior: Helm chart : https://github.com/grafana/helm-charts/tree/main/charts/tempo-distributed custom-values.yaml

tempo: storage: trace: backend: azure block: v2_encoding: zstd azure: container_name: tempotest storage_account_name: test storage_account_key: testkey traces: otlp: http: enabled: true grpc: enabled: true

distributor: service: type: LoadBalancer config: log_received_spans: enabled: true filter_by_status_error: true include_all_attributes: true

I have tried to inject traces data using Job: apiVersion: batch/v1 kind: Job metadata: name: generate-traces spec: template: spec: restartPolicy: Never containers:

• name: tracegen image: Package xk6-client-tracing · GitHub env:
• name: ENDPOINT value: tempo-distributor.observability.svc.cluster.local:4317

I have also used apis to test it gave partial success as result

{“partialSuccess”:{}}

curl -1 -X POST -H ‘Content-Type: application/json’ http://tempo-distributor.observability.svc.cluster.local:4318/v1/traces -d ’ { “resourceSpans”: [{ “resource”: { “attributes”: [{ “key”: “service.name”, “value”: { “stringValue”: “my.service” } }] }, “scopeSpans”: [{ “scope”: { “name”: “my.library”, “version”: “1.0.0”, “attributes”: [{ “key”: “my.scope.attribute”, “value”: { “stringValue”: “some scope attribute” } }] }, “spans”: [ { “traceId”: “5B8EFFF798038103D269B633813FC700”, “spanId”: “EEE19B7EC3C1B100”, “name”: “I am a span!”, “startTimeUnixNano”: 1689969302000000000, “endTimeUnixNano”: 1689970000000000000, “kind”: 2, “attributes”: [ { “key”: “my.span.attr”, “value”: { “stringValue”: “some value” } }] }] }] }] }’ Expected behavior Traces data should be visible

Environment:

• aks
• helm

Additional Context I need to inject traces from dotnet service to tempodb using http endpoint on 4318 port