OTEP: Pluggable Authentication Interface for OTLP Exporters in SDKs

[srprash]

trafficstars

Hi all, Following discussions in recent OpenTelemetry SIG meetings regarding the need for supporting authentication plugins in SDK exporters, it was agreed that a formal proposal should be developed to standardize this capability across languages.

This requirement has been brought up in the past via multiple issues in defferent forms:

• Limited OTLP exporter request authentication methods
• Exporter authentication
• Sigv4 Authentication in OtlpHttp{Signal}Exporter
• Improved support to enable Authentication on otlp exporters (or any exporter?)
• Better support for OTLP header authentication

This PR introduces an OTEP that proposes a pluggable authenticator interface for OTLP exporters within OpenTelemetry SDKs. The goal is to enable users to authenticate telemetry export requests without requiring custom exporter implementations. I’m opening this PR to initiate a broader discussion within the community and to collaboratively shape the direction of this feature.

I’ll update the description with additional context or background links as the discussion evolves. Looking forward to your feedback and contributions.

[srprash]

@DylanRussell @pellared I believe you bring up a common concern where some of the SDKs already have a way (although not a direct one) to achieve authentication either via custom sessions, clients, or something else that can be plugged into the OTLP exporters. The concern this OTEP brings up is that these mechanisms are not present in all the SDKs. And if these SDKs are to add something similar, there is no standard to follow. Should the exporters in these SDKs be accepting sessions or clients or both or something else that makes sense for a language. And then there is also the question on how to make these configurations available to zero-code instrumentation.

Also, we aim to keep the proposed change backwards compatible. So existing users of Google's custom session can continue to use it or switch to using the new authentication plugin if Google decides to build one in the future.

[pellared]

The concern this OTEP brings up is that these mechanisms are not present in all the SDKs.

Could you also cover the SDKs that already have a way (although not a direct one) to achieve authentication? It would be good to call out whether they should also implement something. It would be also good if other SDKs can also provide similar mechanism instead of providing a feature that only covers authentication.

[DylanRussell]

The concern this OTEP brings up is that these mechanisms are not present in all the SDKs. And if these SDKs are to add something similar, there is no standard to follow. Should the exporters in these SDKs be accepting sessions or clients or both or something else that makes sense for a language.

I think it is a good idea to mandate that each exporter provide a way that vendors/users can use to customize auth, but is it critical that they all do it the same way, or is it OK for each to come up with it's own slightly different mechanism ? For example for python, the HTTP exporter could just be updated to accept a requests.auth.AuthBase object/superclass, and use that if provided.

For gRPC there is a standard auth plugin that exists across languages: https://grpc.io/docs/guides/auth/#extending-grpc-to-support-other-authentication-mechanisms . It is not something that each language makes configurable yet (see https://github.com/open-telemetry/opentelemetry-cpp/pull/3403), so maybe what this OTEP should say for gRPC is that ChannelCredentials must be something that can be based into the gRPC exporters, and that we need to figure out how it can be supplied when auto instrumentation is used ?

[MrAlias]

This is introducing a very large security vulnerability target. We are going to maintain clients with elevated permissions to sensitive endpoints.

Does OpenTelemetry want to focus its efforts on maintaining this complex and highly sensitive piece of code? Can we rely on existing solutions within languages instead?

At a minimum there needs to be effort and documented thought put into the security implications of this proposal.

[github-actions[bot]]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[github-actions[bot]]

Closed as inactive. Feel free to reopen if this PR is still being worked on.

[srprash]

Hey. Sorry folks. I got sidelined with other work. I still do want to work on this and hopefully will be able to make time in this week or next. Please reopen this PR. TY.

[github-actions[bot]]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[srprash]

Not stale. :)

[github-actions[bot]]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[srprash]

Hi all. Thanks for the comments on this proposal. Really helpful. Looking at this with fresh eyes, I see that:

1. It is really difficult to define an interface that can be applied to each language SDK and can handle different auth schemes. As suggested by @pellared, it makes more sense to specify the capabilities that the SDKs must support for users to achieve their desired auth mechanisms.
2. Considering that there are existing patterns in some language SDKs (for example, request.Session param to exporters) which can be leveraged to make auth work (as @DylanRussell mentioned), the proposal should be to follow this pattern (or something more suitable for each language) in all the SDKs that do not currently offer this mechanism.
3. As @dashpole suggested, we can focus only on HTTP auth capabilities in this proposal. gRPC seems to have it mostly figured out and I don't have the expertise needed to cover gRPC auth in this proposal's scope.
4. As pointed out by @MrAlias , if Opentelemetry were to maintain some auth components there is a big security burden. However, if OTel SDKs were to let a custom component for auth be plugged in to the exporter by the users themselves, then this responsibility is on the component developer and its users.

Let me know if this makes sense and if I have missed something. I can work on updating the proposal if we agree on the above points.

[github-actions[bot]]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[srprash]

Hey folks. I think the simplest capability that could address all the auth related needs is that SDKs allow users to inject custom HTTP or gRPC clients to the OTLP exporters. Go and Python (in a way) already do this but it may not be feasible for languages such as Java where clients are abstracted away and cannot be exposed. I plan to take a more focused approach per language to get some auth solution working, at least for SigV4 which is my sole use case right now. Thank you all for great feedbacks which helped me learn a lot about authentication and its challenges. :)

[github-actions[bot]]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[pellared]

@srprash, is your plan to have this PR closed or maybe you want to come up with some OTEP or specification changes (e.g. by adding some notes) that summarizes the outcome around the discussion we all had: https://github.com/open-telemetry/opentelemetry-specification/pull/4500#issuecomment-3095267453?

[github-actions[bot]]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[github-actions[bot]]

Closed as inactive. Feel free to reopen if this PR is still being worked on.