opentelemetry-lambda icon indicating copy to clipboard operation
opentelemetry-lambda copied to clipboard

Lambda Traces are not flowing to custom otel aggregator

Open surola opened this issue 3 years ago • 1 comments

Attached the OpenTelemetry Collector AWS Lambda Extension layer to my lambda and routing the traces to custom otel aggregator, Seeing now logs flowing to aggregator

Able to see the traces in console but not flowing to aggregator

collector.yaml used:

receivers: otlp: protocols: grpc: http:

exporters: logging: loglevel: debug otlp: endpoint: agregatordns:4317

service: telemetry: logs: level: debug pipelines:
traces: receivers: [otlp] exporters: [logging, otlp] metrics: receivers: [otlp] exporters: [logging, otlp]

getting the below message apgrpc.go:174 [core] pickfirstBalancer: UpdateSubConnState: 0xc00026d270, {CONNECTING } {"grpc_log": true} 2022-06-22T17:34:48.113Z info zapgrpc/zapgrpc.go:174 [core] [Channel #1] Channel Connectivity change to CONNECTING {"grpc_log": true} 2022-06-22T17:34:48.113Z info zapgrpc/zapgrpc.go:174 [core] [Channel #3 SubChannel #4] Subchannel Connectivity change to CONNECTING {"grpc_log": true} 2022-06-22T17:34:48.113Z info zapgrpc/zapgrpc.go:174 [core] [Channel #3 SubChannel #4] Subchannel picks a new address "agregatordns:4317" to connect {"grpc_log": true} 2022-06-22T17:34:48.113Z info zapgrpc/zapgrpc.go:174 [core] pickfirstBalancer: UpdateSubConnState: 0xc00026dc40, {CONNECTING } {"grpc_log": true} 2022-06-22T17:34:48.113Z info zapgrpc/zapgrpc.go:174 [core] [Channel #3] Channel Connectivity change to CONNECTING {"grpc_log": true} 2022-06-22T17:34:48.113Z info builder/exporters_builder.go:48 Exporter started. {"kind": "exporter", "name": "otlp"} 2022-06-22T17:34:48.113Z info builder/exporters_builder.go:40 Exporter is starting... {"kind": "exporter", "name": "logging"} 2022-06-22T17:34:48.113Z info builder/exporters_builder.go:48 Exporter started. {"kind": "exporter", "name": "logging"} 2022-06-22T17:34:48.113Z info service/service.go:86 Starting processors... 2022-06-22T17:34:48.113Z info builder/pipelines_builder.go:54 Pipeline is starting... {"kind": "pipeline", "name": "metrics"} 2022-06-22T17:34:48.113Z info builder/pipelines_builder.go:65 Pipeline is started. {"kind": "pipeline", "name": "metrics"} 2022-06-22T17:34:48.113Z info builder/pipelines_builder.go:54 Pipeline is starting... {"kind": "pipeline", "name": "traces"} 2022-06-22T17:34:48.113Z info builder/pipelines_builder.go:65 Pipeline is started. {"kind": "pipeline", "name": "traces"} 2022-06-22T17:34:48.113Z info service/service.go:91 Starting receivers... 2022-06-22T17:34:48.113Z info builder/receivers_builder.go:68 Receiver is starting... {"kind": "receiver", "name": "otlp"} 2022-06-22T17:34:48.113Z info zapgrpc/zapgrpc.go:174 [core] [Server #5] Server created {"grpc_log": true} 2022-06-22T17:34:48.113Z info otlpreceiver/otlp.go:70 Starting GRPC server on endpoint 0.0.0.0:4317 {"kind": "receiver", "name": "otlp"} 2022-06-22T17:34:48.114Z info otlpreceiver/otlp.go:88 Starting HTTP server on endpoint 0.0.0.0:4318 {"kind": "receiver", "name": "otlp"} 2022-06-22T17:34:48.114Z info builder/receivers_builder.go:73 Receiver started. {"kind": "receiver", "name": "otlp"} 2022-06-22T17:34:48.114Z info service/collector.go:251 Starting otelcol... {"Version": "v0.1.0", "NumCPU": 2} 2022-06-22T17:34:48.114Z info service/collector.go:146 Everything is ready. Begin running and processing data. 2022-06-22T17:34:48.115Z info zapgrpc/zapgrpc.go:174 [core] [Server #5 ListenSocket #6] ListenSocket created {"grpc_log": true} {'secret_id': 'cdp-twistlock-install-bundle-test', 'region': 'us-west-2'} EXTENSION Name: collector State: Ready Events: [INVOKE,SHUTDOWN] 2022/06/22 17:34:48 [collector] Received event: { "eventType": "INVOKE", "deadlineMs": 1655919468416, "requestId": "c631bc9e-77d3-41b4-b3f9-f709322cc60d", "invokedFunctionArn": "arn:aws:lambda:us-west-2:account:function:test-lambda", "tracing": { "type": "X-Amzn-Trace-Id", "value": "Root=1-62b352b7-2e9e993970bdd594507dcbfc;Parent=4158ddb7404d0c14;Sampled=0" } } 2022/06/22 17:34:48 [collector] Waiting for event... arn:aws:secretsmanager:us-west-2:account:secret:bundle-test-94oCRJ:INSTALL_BUNDLE:: END RequestId: c631bc9e-77d3-41b4-b3f9-f709322cc60d REPORT RequestId: c631bc9e-77d3-41b4-b3f9-f709322cc60d Duration: 156.44 ms Billed Duration: 157 ms Memory Size: 1088 MB Max Memory Used: 91 MB Init Duration: 491.94 ms

surola avatar Jun 22 '22 19:06 surola