cp-docker-images icon indicating copy to clipboard operation
cp-docker-images copied to clipboard

Kafka connec to push to event hubs

Open chinwobble opened this issue 4 years ago • 0 comments

I am trying to connect kafka connect to event hubs based on information in this guide. https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-kafka-connect-tutorial

I've tried to connect but I get the error;

Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment

Here is my docker-compose.yml file:

---
version: "2"
services:
  default-config:
    image: confluentinc/cp-kafka-connect:latest
    stdin_open: true
    tty: true
    environment:
      CONNECT_BOOTSTRAP_SERVERS: my-namespace.servicebus.windows.net:9093
      CONNECT_GROUP_ID: "connect-cluster-group"
      CONNECT_CONFIG_STORAGE_TOPIC: "connect-cluster-config"
      CONNECT_OFFSET_STORAGE_TOPIC: "connect-cluster-offsets"
      CONNECT_STATUS_STORAGE_TOPIC: "connect-cluster-status"
      # internal topic replication factors - auto 3x replication in Azure Storage
      CONFIG_STORAGE_REPLICATION_FACTOR: 1
      OFFSET_STORAGE_REPLICATION_FACTOR: 1
      STATUS_STORAGE_REPLICATION_FACTOR: 1
      
      REST_ADVERTISED_HOST_NAME: CONNECT
      OFFSET_FLUSH_INTERVAL_MS: 10000
      CONNECT_KEY_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
      CONNECT_VALUE_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
      CONNECT_INTERNAL_KEY_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
      CONNECT_INTERNAL_VALUE_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
      CONNECT_REST_ADVERTISED_HOST_NAME: "default-config"
      # required EH Kafka security settings
      CONNECT_SECURITY.PROTOCOL: "SASL_SSL"
      CONNECT_SASL_MECHANISM: "PLAIN"
      CONNECT_SASL_JAAS_CONFIG: $CONN_STRING

      CONNECT_PRODUCER_SECURITY_PROTOCOL: "SASL_SSL"
      CONNECT_PRODUCER_SASL_MECHANISM: "PLAIN"

      CONNECT_PRODUCER_SASL_JAAS_CONFIG: $CONN_STRING

      CONNECT_CONSUMER_SECURITY_PROTOCOL: "SASL_SSL"
      CONNECT_CONSUMER_SASL_MECHANISM: "PLAIN"

      CONNECT_CONSUMER_SASL_JAAS_CONFIG: $CONN_STRING
      CONNECT_REST_ADVERTISED_HOST_NAME: "localhost"

chinwobble avatar Oct 22 '20 13:10 chinwobble