functions-samples icon indicating copy to clipboard operation
functions-samples copied to clipboard

[DOCS] for sample: https://github.com/firebase/functions-samples/tree/main/2nd-gen/instrument-with-opentelemetry

Open dan-wu-open opened this issue 2 years ago • 7 comments

Which sample?

https://github.com/firebase/functions-samples/tree/main/2nd-gen/instrument-with-opentelemetry

What is the issue with this sample's docs?

The docs mention a ./.env file but I don't see it. Would appreciate it to see how it's set up! Would love to add telemetry to all of our cloud functions

dan-wu-open avatar Oct 20 '22 19:10 dan-wu-open

Yeah been trying to set this up using NODE_OPTIONS but can't get it working. @taeold Really excited about this but would love your input! 🙏

adamnathanlewis avatar Oct 20 '22 21:10 adamnathanlewis

By the way @dan-wu-open I think you might be able to get what you're looking for by setting an environment variable called NODE_OPTIONS to --require "./tracing.js". You can set this with an env file or as part of your CI/CD or manually on the function within the google cloud console. That said, it isn't working for me! Worth giving it a try though

https://console.cloud.google.com/functions/list Select your function Edit Expand Runtime, build, connections and security settings

image

Give it a try 😄

adamnathanlewis avatar Oct 21 '22 09:10 adamnathanlewis

I am struggling with setting this up in TypeScript, somehow my custom spans are not visible. One general remark useful while testing: by default Cloud Run will trace 0.1 request per second per container (https://cloud.google.com/run/docs/trace#trace_sampling_rate). When I was invoking my callable from Angular app the OPTIONS preflight request was draining this quota and real request were not traced (there is nice property in logs traceSampled: true saying if given request was sampled or not)

kaczors avatar Oct 21 '22 12:10 kaczors

Whoops my mistake. Thanks for calling it out @dan-wu-open and @adamaffinitech. Fix incoming at https://github.com/firebase/functions-samples/pull/1022

taeold avatar Oct 25 '22 22:10 taeold

@kaczors Thanks for sharing your problem.

It's a real bummer that we don't get to configure the Cloud Run sampling rate :(.

I believe Open Telemetry allows you to configure a sampler that can override the 0.1 sampling rate of Cloud Run. See https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#built-in-samplers.

Unfortunately, this would mean that you won't have the overarching request span generated by the Cloud Run's load balancer, but you'd at least be getting spans generated by your application.

I will take a closer look and see if I can write some sample code this weekend. If you are willing to share your work, I think the community will appreciate it!

taeold avatar Oct 25 '22 22:10 taeold

@kaczors Thanks for sharing your problem.

It's a real bummer that we don't get to configure the Cloud Run sampling rate :(.

I believe Open Telemetry allows you to configure a sampler that can override the 0.1 sampling rate of Cloud Run. See https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#built-in-samplers.

Unfortunately, this would mean that you won't have the overarching request span generated by the Cloud Run's load balancer, but you'd at least be getting spans generated by your application.

I will take a closer look and see if I can write some sample code this weekend. If you are willing to share your work, I think the community will appreciate it!

Do you happen to have any working code around this?

I'm using v2 callable functions but only the OPTIONS pre-flight CORS request is getting its trace sampled. The actual onCall function invocation gets a traceId and spanId, but I think due to the sampling rate of Cloud Run, it's not then getting sampled and showing up in the trace explorer.

georgecartridge avatar Jan 23 '24 16:01 georgecartridge

@kaczors Thanks for sharing your problem. It's a real bummer that we don't get to configure the Cloud Run sampling rate :(. I believe Open Telemetry allows you to configure a sampler that can override the 0.1 sampling rate of Cloud Run. See https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#built-in-samplers. Unfortunately, this would mean that you won't have the overarching request span generated by the Cloud Run's load balancer, but you'd at least be getting spans generated by your application. I will take a closer look and see if I can write some sample code this weekend. If you are willing to share your work, I think the community will appreciate it!

Do you happen to have any working code around this?

I'm using v2 callable functions but only the OPTIONS pre-flight CORS request is getting its trace sampled. The actual onCall function invocation gets a traceId and spanId, but I think due to the sampling rate of Cloud Run, it's not then getting sampled and showing up in the trace explorer.

sorry, I did not found any workaround and not using this solution anymore

kaczors avatar Jan 25 '24 07:01 kaczors