opentelemetry-collector-contrib icon indicating copy to clipboard operation
opentelemetry-collector-contrib copied to clipboard

[extension/headers_setter] Add headers_setter extension.

Open kovrus opened this issue 3 years ago • 2 comments

Description: <Describe what has changed.>

The headers_setter extension implements configauth.GRPCClientAuthenticator and is used to set request headers in gRPC / HTTP exporters with values provided via the extension configuration or request metadata (context).

Use cases include but are not limited to enabling multi-tenancy for observability backends such as Tempo, Mimir, Loki and others by setting the X-Scope-OrgID header to the value extracted from the context.

Link to tracking Issue: <Issue number if applicable> This change potentially can address the following issues related to multi-tenancy use cases:

  • https://github.com/open-telemetry/opentelemetry-collector/issues/5733

  • https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/7945

  • https://github.com/open-telemetry/opentelemetry-collector/issues/4814

  • https://github.com/open-telemetry/opentelemetry-collector/issues/4806

  • https://github.com/open-telemetry/opentelemetry-collector/issues/5264

  • We can drop specific multi-tenant implementation in Loki https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/12415

Open discussion:

  • At the moment, this extension cannot be used with the batch processor, see https://github.com/open-telemetry/opentelemetry-collector/issues/4544
  • The extension is implemented as the client type authenticator, so it uses auth and authenticator config to be enabled which might be confusing. One option could be renaming auth -> interception and authenticator -> interceptor cc: @jpkrohling
...
exporters:
...
  loki:
    ...
    auth:
      authenticator: headers_setter  
...

Alternatives: It is also possible to implement the same functionality (setting header values from context) in opentelemetry-collector by implementing a round tripper and interceptor that will set in confighttp.go and configgrpc.go and changing the headers configuration of HTTP / gRPC client settings. This approach seems to be more complex and might impact some exporters in pentelemetry-collector-contrib

Testing: <Describe what testing was performed and which tests were added.>

  • Unit tests
  • Locally with the following Otel collector config:
extensions:
  headers_setter:
    headers:
      key: X-Scope-OrgID
      from_context: "tenant"

receivers:
  otlp/1:
    protocols:
      grpc:
        include_metadata: true
        endpoint: localhost:4000
  otlp/2:
    protocols:
      grpc:
        endpoint: localhost:4318
        include_metadata: true

exporters:
  otlp:
    endpoint: localhost:4318
    tls:
      insecure: true
      insecure_skip_verify: true
    headers:
      tenant: "value1"
  logging:

  loki:
    labels:
      resource:
        container_name: ""
        container_id: ""
    endpoint: https://localhost:3100/loki/api/v1/push
    auth:
      authenticator: headers_setter

service:
  extensions: [headers_setter]
  pipelines:
    logs/1:
      receivers: [otlp/1]
      exporters: [otlp]
    logs/2:
      receivers: [otlp/2]
      exporters: [loki]

Documentation: <Describe the documentation added.> See README.md in the new headers_setter extension.

kovrus avatar Aug 02 '22 15:08 kovrus

CLA Signed

The committers listed above are authorized under a signed CLA.

  • :white_check_mark: login: kovrus / name: Ruslan Kovalov (e01947098cbb0140bcfd437e2a1a71f3ef7f184c)

@kovrus, please open an issue proposing this new component (see the contributing guidelines). I'm willing to sponsor this one.

jpkrohling avatar Aug 02 '22 19:08 jpkrohling