opentelemetry-collector-contrib icon indicating copy to clipboard operation
opentelemetry-collector-contrib copied to clipboard

chore: [receiver/kafka] docs add example of connecting to kafka with sasl +tls

Open povilasv opened this issue 1 year ago • 4 comments

Description:

Adding an example on how to connect to kafka that needs both tls and sasl config.

Link to tracking Issue: <Issue number if applicable> https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/31931

Testing: <Describe what testing was performed and which tests were added.>

Documentation: <Describe the documentation added.>

povilasv avatar Apr 25 '24 11:04 povilasv

@povilasv @MovieStoreGuy What do you think about this? I would like to stay consistent with the OTLP receiver, which does:

receivers:
  otlp:
    protocols:
      grpc:

So we could just have it such that setting tls enables it. But we could also have a tls_enabled key, or try to put an enabled field in the tls section.

evan-bradley avatar Apr 29 '24 20:04 evan-bradley

@evan-bradley updated PR based on comments.

I've tried using:

  kafka:
    topic: meh
    brokers: localhost:9092
    auth:
      tls:

And it' seems that it renders tls section to nil and doesn't enable TLS.

So for now the only way is to set some field under auth :thinking:

povilasv avatar Apr 30 '24 08:04 povilasv

Sorry, I lost track of this.

And it' seems that it renders tls section to nil and doesn't enable TLS.

My bad, I should have been a bit clearer: I think we can update the code to work more like the OTLP receiver to allow merely setting the tls key to mean "enable TLS" by creating a default config, then setting it to nil if the user doesn't specify the key. Here's how the OTLP receiver does each:

  1. Creating a default config: https://github.com/open-telemetry/opentelemetry-collector/blob/main/receiver/otlpreceiver/factory.go#L41
  2. The default config is kept if the user sets the key, otherwise set to nil and disabled if they do not: https://github.com/open-telemetry/opentelemetry-collector/blob/main/receiver/otlpreceiver/config.go#L68

evan-bradley avatar May 15 '24 13:05 evan-bradley

@povilasv @MovieStoreGuy What do you think about this? I would like to stay consistent with the OTLP receiver, which does:

receivers:
  otlp:
    protocols:
      grpc:

So we could just have it such that setting tls enables it. But we could also have a tls_enabled key, or try to put an enabled field in the tls section.

The config needs some love since it is somewhat of a sprawling mess as a side affect of how Kafka library exposes its configuration. I agree it needs some love, but I don't want to do a bunch of subtle changes that makes it hard to understand what changed.

Adding an example as an interim solution is acceptable until we can plan this out.

MovieStoreGuy avatar May 22 '24 02:05 MovieStoreGuy

The config needs some love since it is somewhat of a sprawling mess as a side affect of how Kafka library exposes its configuration. I agree it needs some love, but I don't want to do a bunch of subtle changes that makes it hard to understand what changed.

Adding an example as an interim solution is acceptable until we can plan this out.

I agree, I tried implementing it, but it's a bit hard. I think we should add the doc, so that users don't spend time debugging this and then later as a separate issue to figure out how to refactor this

povilasv avatar May 22 '24 14:05 povilasv