opentelemetry-go-instrumentation
opentelemetry-go-instrumentation copied to clipboard
Published
20 hours ago •
open-telemetry
open-telemetry
No Trace in jaeger
In the file https://github.com/open-telemetry/opentelemetry-go-instrumentation/blob/main/docs/tutorial/emojivoto-instrumented.yaml uses the image otel-go-instrumentation; Howerver, it occurs error pull images. So , I follow the doc getting-started.md and change the images otel/autoinstrumentation-go, but there is no trace in the jaeger.
apiVersion: v1
kind: Namespace
metadata:
name: emojivoto
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: emoji
namespace: emojivoto
labels:
app.kubernetes.io/name: emoji
app.kubernetes.io/part-of: emojivoto
app.kubernetes.io/version: v11
spec:
replicas: 1
selector:
matchLabels:
app: emoji-svc
version: v11
template:
metadata:
labels:
app: emoji-svc
version: v11
spec:
shareProcessNamespace: true
containers:
- env:
- name: GRPC_PORT
value: "8080"
- name: PROM_PORT
value: "8801"
image: docker.l5d.io/buoyantio/emojivoto-emoji-svc:v11
name: emoji-svc
ports:
- containerPort: 8080
name: grpc
- containerPort: 8801
name: prom
resources:
requests:
cpu: 100m
- name: emojivoto-emoji-instrumentation
image: otel/autoinstrumentation-go
imagePullPolicy: IfNotPresent
env:
- name: OTEL_GO_AUTO_TARGET_EXE
value: /usr/local/bin/emojivoto-emoji-svc
- name: OTEL_EXPORTER_OTLP_ENDPOINT
value: "http://jaeger:4318"
- name: OTEL_SERVICE_NAME
value: "emojivoto-emoji"
securityContext:
runAsUser: 0
privileged: true
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: voting
namespace: emojivoto
labels:
app.kubernetes.io/name: voting
app.kubernetes.io/part-of: emojivoto
app.kubernetes.io/version: v11
spec:
replicas: 1
selector:
matchLabels:
app: voting-svc
version: v11
template:
metadata:
labels:
app: voting-svc
version: v11
spec:
shareProcessNamespace: true
containers:
- env:
- name: GRPC_PORT
value: "8080"
- name: PROM_PORT
value: "8801"
image: docker.l5d.io/buoyantio/emojivoto-voting-svc:v11
name: voting-svc
ports:
- containerPort: 8080
name: grpc
- containerPort: 8801
name: prom
resources:
requests:
cpu: 100m
- name: emojivoto-voting-instrumentation
image: otel/autoinstrumentation-go
imagePullPolicy: IfNotPresent
env:
- name: OTEL_GO_AUTO_TARGET_EXE
value: /usr/local/bin/emojivoto-voting-svc
- name: OTEL_EXPORTER_OTLP_ENDPOINT
value: "http://jaeger:4318"
- name: OTEL_SERVICE_NAME
value: "emojivoto-voting"
securityContext:
runAsUser: 0
privileged: true
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: web
namespace: emojivoto
labels:
app.kubernetes.io/name: web
app.kubernetes.io/part-of: emojivoto
app.kubernetes.io/version: v11
spec:
replicas: 1
selector:
matchLabels:
app: web-svc
version: v11
template:
metadata:
labels:
app: web-svc
version: v11
spec:
shareProcessNamespace: true
containers:
- env:
- name: WEB_PORT
value: "8080"
- name: EMOJISVC_HOST
value: emoji-svc.emojivoto:8080
- name: VOTINGSVC_HOST
value: voting-svc.emojivoto:8080
- name: INDEX_BUNDLE
value: dist/index_bundle.js
image: docker.l5d.io/buoyantio/emojivoto-web:v11
name: web-svc
ports:
- containerPort: 8080
name: http
resources:
requests:
cpu: 100m
- name: emojivoto-web-instrumentation
image: otel/autoinstrumentation-go
imagePullPolicy: IfNotPresent
env:
- name: OTEL_GO_AUTO_TARGET_EXE
value: /usr/local/bin/emojivoto-web
- name: OTEL_EXPORTER_OTLP_ENDPOINT
value: "http://jaeger:4318"
- name: OTEL_SERVICE_NAME
value: "emojivoto-web"
securityContext:
runAsUser: 0
privileged: true
---
apiVersion: v1
kind: Service
metadata:
name: web-svc
namespace: emojivoto
spec:
type: ClusterIP
selector:
app: web-svc
ports:
- name: http
port: 80
targetPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: voting-svc
namespace: emojivoto
spec:
selector:
app: voting-svc
ports:
- name: grpc
port: 8080
targetPort: 8080
- name: prom
port: 8801
targetPort: 8801
---
apiVersion: v1
kind: Service
metadata:
name: emoji-svc
namespace: emojivoto
spec:
selector:
app: emoji-svc
ports:
- name: grpc
port: 8080
targetPort: 8080
- name: prom
port: 8801
targetPort: 8801
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: vote-bot
namespace: emojivoto
labels:
app.kubernetes.io/name: vote-bot
app.kubernetes.io/part-of: emojivoto
app.kubernetes.io/version: v11
spec:
replicas: 1
selector:
matchLabels:
app: vote-bot
version: v11
template:
metadata:
labels:
app: vote-bot
version: v11
spec:
containers:
- command:
- emojivoto-vote-bot
env:
- name: WEB_HOST
value: web-svc.emojivoto:80
image: docker.l5d.io/buoyantio/emojivoto-web:v11
name: vote-bot
resources:
requests:
cpu: 10m
The logs of the instrumentaion is below:
root@dell-1080:/home/dell/tyz# kubectl logs pod/emoji-57bb598648-qdvsf -n emojivoto -c emojivoto-emoji-instrumentation -f
{"time":"2025-03-06T02:49:24.964989158Z","level":"INFO","source":{"function":"main.main","file":"/app/cli/main.go","line":103},"msg":"building OpenTelemetry Go instrumentation ...","globalImpl":false,"version":{"Release":"v0.21.0","Revision":"unknown","Go":{"Version":"go1.24.0","OS":"linux","Arch":"amd64"}}}
{"time":"2025-03-06T02:49:26.967055515Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto/internal/pkg/process.(*Analyzer).DiscoverProcessID","file":"/app/internal/pkg/process/discover.go","line":64},"msg":"found process","pid":11}
{"time":"2025-03-06T02:49:26.973520894Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto/internal/pkg/process.remoteAllocate.func1","file":"/app/internal/pkg/process/allocate.go","line":76},"msg":"Detaching from process","pid":11}
{"time":"2025-03-06T02:49:26.973731843Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto.NewInstrumentation","file":"/app/instrumentation.go","line":110},"msg":"target process analysis completed","pid":11,"go_version":"1.15.0","dependencies":{"contrib.go.opencensus.io/exporter/ocagent":"0.6.0","github.com/beorn7/perks":"1.0.1","github.com/census-instrumentation/opencensus-proto":"0.2.1","github.com/cespare/xxhash/v2":"2.1.1","github.com/golang/groupcache":"0.0.0-20200121045136-8c9f03a8e57e","github.com/golang/protobuf":"1.4.0","github.com/grpc-ecosystem/go-grpc-prometheus":"1.2.0","github.com/grpc-ecosystem/grpc-gateway":"1.14.4","github.com/matttproud/golang_protobuf_extensions":"1.0.1","github.com/prometheus/client_golang":"1.6.0","github.com/prometheus/client_model":"0.2.0","github.com/prometheus/common":"0.9.1","github.com/prometheus/procfs":"0.0.11","go.opencensus.io":"0.22.3","golang.org/x/net":"0.0.0-20200425230154-ff2c4b7c35a0","golang.org/x/sync":"0.0.0-20200317015054-43a5402ce75a","golang.org/x/sys":"0.0.0-20200430082407-1f5687305801","golang.org/x/text":"0.3.2","google.golang.org/api":"0.22.0","google.golang.org/genproto":"0.0.0-20200430143042-b979b6f78d84","google.golang.org/grpc":"1.29.1","google.golang.org/protobuf":"1.21.0","std":"1.15.0"},"total_functions_found":9}
{"time":"2025-03-06T02:49:26.977066125Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto/internal/pkg/instrumentation.(*Manager).loadProbes","file":"/app/internal/pkg/instrumentation/manager.go","line":374},"msg":"loading probe","name":{"SpanKind":2,"InstrumentedPkg":"google.golang.org/grpc"}}
{"time":"2025-03-06T02:49:26.978087124Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto/internal/pkg/instrumentation/probe.StructFieldConst.InjectOption","file":"/app/internal/pkg/instrumentation/probe/probe.go","line":494},"msg":"Offset not cached, analyzing directly","key":"TCPAddr_IP_offset","id":{"ModPath":"std","PkgPath":"net","Struct":"TCPAddr","Field":"IP"}}
{"time":"2025-03-06T02:49:27.043292191Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto/internal/pkg/instrumentation/probe.StructFieldConst.InjectOption","file":"/app/internal/pkg/instrumentation/probe/probe.go","line":494},"msg":"Offset not cached, analyzing directly","key":"TCPAddr_Port_offset","id":{"ModPath":"std","PkgPath":"net","Struct":"TCPAddr","Field":"Port"}}
{"time":"2025-03-06T02:49:27.252336253Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto/internal/pkg/instrumentation.(*Manager).loadProbes","file":"/app/internal/pkg/instrumentation/manager.go","line":374},"msg":"loading probe","name":{"SpanKind":2,"InstrumentedPkg":"net/http"}}
{"time":"2025-03-06T02:49:27.252928261Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto/internal/pkg/instrumentation/probe.StructFieldConst.InjectOption","file":"/app/internal/pkg/instrumentation/probe/probe.go","line":494},"msg":"Offset not cached, analyzing directly","key":"method_ptr_pos","id":{"ModPath":"std","PkgPath":"net/http","Struct":"Request","Field":"Method"}}
{"time":"2025-03-06T02:49:27.315191671Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto/internal/pkg/instrumentation/probe.StructFieldConst.InjectOption","file":"/app/internal/pkg/instrumentation/probe/probe.go","line":494},"msg":"Offset not cached, analyzing directly","key":"url_ptr_pos","id":{"ModPath":"std","PkgPath":"net/http","Struct":"Request","Field":"URL"}}
{"time":"2025-03-06T02:49:27.375526173Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto/internal/pkg/instrumentation/probe.StructFieldConst.InjectOption","file":"/app/internal/pkg/instrumentation/probe/probe.go","line":494},"msg":"Offset not cached, analyzing directly","key":"ctx_ptr_pos","id":{"ModPath":"std","PkgPath":"net/http","Struct":"Request","Field":"ctx"}}
{"time":"2025-03-06T02:49:27.433244178Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto/internal/pkg/instrumentation/probe.StructFieldConst.InjectOption","file":"/app/internal/pkg/instrumentation/probe/probe.go","line":494},"msg":"Offset not cached, analyzing directly","key":"path_ptr_pos","id":{"ModPath":"std","PkgPath":"net/url","Struct":"URL","Field":"Path"}}
{"time":"2025-03-06T02:49:27.490879401Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto/internal/pkg/instrumentation/probe.StructFieldConst.InjectOption","file":"/app/internal/pkg/instrumentation/probe/probe.go","line":494},"msg":"Offset not cached, analyzing directly","key":"headers_ptr_pos","id":{"ModPath":"std","PkgPath":"net/http","Struct":"Request","Field":"Header"}}
{"time":"2025-03-06T02:49:27.549069212Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto/internal/pkg/instrumentation/probe.StructFieldConst.InjectOption","file":"/app/internal/pkg/instrumentation/probe/probe.go","line":494},"msg":"Offset not cached, analyzing directly","key":"req_ptr_pos","id":{"ModPath":"std","PkgPath":"net/http","Struct":"response","Field":"req"}}
{"time":"2025-03-06T02:49:27.609872111Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto/internal/pkg/instrumentation/probe.StructFieldConst.InjectOption","file":"/app/internal/pkg/instrumentation/probe/probe.go","line":494},"msg":"Offset not cached, analyzing directly","key":"status_code_pos","id":{"ModPath":"std","PkgPath":"net/http","Struct":"response","Field":"status"}}
{"time":"2025-03-06T02:49:27.72320988Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto/internal/pkg/instrumentation/probe.StructFieldConst.InjectOption","file":"/app/internal/pkg/instrumentation/probe/probe.go","line":494},"msg":"Offset not cached, analyzing directly","key":"remote_addr_pos","id":{"ModPath":"std","PkgPath":"net/http","Struct":"Request","Field":"RemoteAddr"}}
{"time":"2025-03-06T02:49:27.780258063Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto/internal/pkg/instrumentation/probe.StructFieldConst.InjectOption","file":"/app/internal/pkg/instrumentation/probe/probe.go","line":494},"msg":"Offset not cached, analyzing directly","key":"host_pos","id":{"ModPath":"std","PkgPath":"net/http","Struct":"Request","Field":"Host"}}
{"time":"2025-03-06T02:49:27.850964099Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto/internal/pkg/instrumentation/probe.StructFieldConst.InjectOption","file":"/app/internal/pkg/instrumentation/probe/probe.go","line":494},"msg":"Offset not cached, analyzing directly","key":"proto_pos","id":{"ModPath":"std","PkgPath":"net/http","Struct":"Request","Field":"Proto"}}
{"time":"2025-03-06T02:49:28.051861458Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto/internal/pkg/instrumentation.(*Manager).loadProbes","file":"/app/internal/pkg/instrumentation/manager.go","line":374},"msg":"loading probe","name":{"SpanKind":3,"InstrumentedPkg":"google.golang.org/grpc"}}
{"time":"2025-03-06T02:49:28.174281963Z","level":"INFO","source":{"function":"main.main","file":"/app/cli/main.go","line":129},"msg":"instrumentation loaded successfully, starting..."}
{"time":"2025-03-06T02:49:28.190306559Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto/internal/pkg/instrumentation.(*Manager).ConfigLoop","file":"/app/internal/pkg/instrumentation/manager.go","line":240},"msg":"Configuration provider closed, configuration updates will no longer be received"}
It seems everthing is ok? And I merge these yaml files into one yaml as I show you before, Can you use this yaml file of mine and reappear the same question?
I faced the same issue, @MrAlias . I started exploring with zero code v0.20.0, but with Emojivoto, I wasn't receiving any trace data, even though all the containers were running fine. Later, the Span Rolling Dice project worked for me.
I have a similar issue; after using autoinstrumentation-go, some pods have outgoing requests with the Traceparent header while others do not. This is really strange and appears to happen randomly.
Practicly same issue here, i try basicly with latest version of operator, collector and autoinstrumentation and i sent everything to Tempo. Traces arrive empty or internal BS. Cannot find anything proper on this topic
@TanYuzhen can you run with debug logs enabled and post the output after spans have been received.