Allow sasl authentication on kafka sources

[GentleGhostCoder]

It is currently not possible to use a Sasl connection with a Kafka source. Currently you would get the following error message with a SASL configuration:

Command failed: Failed to create Kafka consumer.
Caused by:
    Client creation error: No provider for SASL mechanism SCRAM-SHA-256: recompile librdkafka with libsasl2 or openssl support. Current build options: PLAIN

Example config:

#
      # Kafka source config file.
      #
      source_id: kafka-source
      source_type: kafka
      parameters:
        topic: kafka-source
        client_params:
          bootstrap.servers: "localhost:9092"
          group.id: quickwit
          security.protocol=SASL_SSL
          sasl.mechanism=SCRAM-SHA-256
          sasl.username: test_user
          sasl.password: changeMe

To add the feature, you would have to add the "sasl" feature for the rdkafka dependency in the Cargo.toml file. File: https://github.com/quickwit-oss/quickwit/blob/main/quickwit/quickwit-indexing/Cargo.toml Line: [27] rdkafka = { workspace = true, optional = true } Change: rdkafka = { features = ["ssl", "sasl"], optional = true }

More info can be found here: https://docs.rs/rdkafka/0.6.0/rdkafka/index.html

[guilload]

Hi @semmjon,

Thanks for opening this nicely-written issue. Do you want to open a PR and make your first contribution to Quickwit? :)

[GentleGhostCoder]

I'll do it that gladly :-)

[guilload]

@semmjon, gentle ping :)