camel-kafka-connector icon indicating copy to clipboard operation
camel-kafka-connector copied to clipboard

Issue setting up camel-google-storage-source-kafka-connector

Open jethomaspanw opened this issue 3 years ago • 1 comments

I'm attempting to set up a google storage source connector but am running into an error. As far as I can tell, I've set up the configuration correctly (there are very few options for this connector), so I'm hoping that I can get some advice on how to debug this error.

Helm Chart Yaml

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
  name: {{ .Values.name }}
  namespace: {{ .Values.namespace }}
  labels:
    # The strimzi.io/cluster label identifies the KafkaConnect instance
    # in which to create this connector. That KafkaConnect instance
    # must have the strimzi.io/use-connector-resources annotation
    # set to true.
    strimzi.io/cluster: {{ .Values.kafkaConnect.name }}
spec:
  class: org.apache.camel.kafkaconnector.googlestoragesource.CamelGooglestoragesourceSourceConnector
  tasksMax: {{ .Values.tasksMax }}
  config:

    topics: {{ .Values.topics }}

    camel.kamelet.google-storage-source.bucketNameOrArn: {{ .Values.bucketName }}
    camel.kamelet.google-storage-source.serviceAccountKey: {{ .Values.saKey }}
    camel.kamelet.google-storage-source.deleteAfterRead: {{ .Values.deleteAfterRead }}

Values.yaml

# Default values for google-storage-source-connector.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
# the connector name
name: google-storage-source
namespace: kafka-namespace
kafkaConnect:
  # the Kafka Connect cluster name
  name: my-connect-cluster
tasksMax: 2
topics: gsc

bucketName: XXXX
saKey: ${file:/opt/kafka/external-configuration/google-storage-source-secret/google-storage-source-secret.properties:serviceAccountKey64}
deleteAfterRead: false

StackTrace

org.apache.kafka.connect.errors.ConnectException: Failed to create and start Camel context
              at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:175)
              at org.apache.kafka.connect.runtime.WorkerSourceTask.initializeAndStart(WorkerSourceTask.java:225)
              at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:186)
              at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:243)
              at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
              at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
              at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
              at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
              at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalStateException: Unable to compute endpoint uri. Reason: uri factory for schema `null` not found
  at org.apache.camel.kafkaconnector.utils.TaskHelper.buildUrl(TaskHelper.java:60)
  at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:142)
  ... 8 more

jethomaspanw avatar Apr 28 '22 16:04 jethomaspanw

What camel kafka connect version are you using?

valdar avatar May 01 '22 21:05 valdar