0xPlaygrounds
feat: Otel is too verbose and expensive
- [ ] I have looked for existing issues (including closed) about this
Feature Request
Make the Otel records configurable.
Motivation
Otel is adding a lot of crap to spans which reflect in all the http debug logs after that, quite annoying and can leak PII (specially the messages). It is also costly to do a bunch of JSON serialization all the time (I did partially fix that in #1052)
Proposal
I am not a tracing expert, I don't really know if it is possible to disable recording fields. Otherwise it could be a feature flag.
Alternatives
I guess the consumer can filter those?
![linear[bot] avatar](https://avatars.githubusercontent.com/in/20150?v=4 "Published by a pub.dev verified publisher"){kind=small}
Yeah we should probably add a feature flag or way to configure this. It's been low priority for a while but I think a way to configure this would not go amiss
I will need to dig more on how to do that. I think we might need a dynamic configuration for it per model or something if we want targeted thing.
Had a look into this.
As far as I'm concerned from what I have discovered, the message logging in GenAI SemConv is optional - so we should be safe to remove the message logging from spans without any worry as it adds an extremely high amount of cardinality. We should probably prefer the messages/requests to be in logs, instead.
This is probably the millionth breaking change we've made, but it's likely a correct one given that not everyone (read: almost nobody at all) needs the entire message history in their span telemetry.