camel-kafka-connector icon indicating copy to clipboard operation
camel-kafka-connector copied to clipboard

Invalid URI exception with camel-couchdb-kafka-connector v0.11.5

Open millenc opened this issue 3 years ago • 0 comments

I want to capture changes done on a CouchDB database and publish them as events to Kafka using the camel-couchdb-kafka-source-connector downloaded from here (version 0.11.5).

I've extracted the contents of the tar pakacge on my plugins path and created a couchdb-source.properties configuration file with:

name=CamelCouchdbSourceConnector
connector.class=org.apache.camel.kafkaconnector.couchdb.CamelCouchdbSourceConnector
tasks.max=1

# use the kafka converters that better suit your needs, these are just defaults:
key.converter=org.apache.kafka.connect.storage.StringConverter
value.converter=org.apache.kafka.connect.storage.StringConverter

# comma separated topics to send messages into
topics=mytopic

# CouchDB database configuration (source)
camel.source.path.database=local
camel.source.path.hostname=couchdb
camel.source.path.protocol=http
camel.source.path.port=5984
camel.source.endpoint.username=admin
camel.source.endpoint.password=<redacted>

Both Kafka and CouchDB are running as Docker containers on the same network using the Bitnami images. The CouchDB container is named couchdb, hence the camel.source.path.hostname=couchdb part. If I start the connector with:

connect-standalone.sh /opt/bitnami/kafka/config/connect-standalone.properties /opt/bitnami/kafka/config/couchdb-source.properties

I get the following exception:

[2022-11-02 14:38:05,618] ERROR [CamelCouchdbSourceConnector|task-0] WorkerSourceTask{id=CamelCouchdbSourceConnector-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:196)
org.apache.kafka.connect.errors.ConnectException: Failed to create and start Camel context
	at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:175)
	at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.initializeAndStart(AbstractWorkerSourceTask.java:270)
	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:187)
	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:244)
	at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.run(AbstractWorkerSourceTask.java:72)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.camel.FailedToCreateRouteException: Failed to create route route1: Route(route1)[From[couchdb:http:couchdb:5984/local?password=... because of Failed to resolve endpoint: couchdb://http:couchdb:5984/local?password=xxxxxx due to: Invalid URI. Format must be of the form couchdb:http[s]://hostname[:port]/database?[options...]
	at org.apache.camel.reifier.RouteReifier.createRoute(RouteReifier.java:80)
	at org.apache.camel.impl.DefaultModelReifierFactory.createRoute(DefaultModelReifierFactory.java:49)
	at org.apache.camel.impl.DefaultCamelContext.startRouteDefinitions(DefaultCamelContext.java:826)
	at org.apache.camel.impl.DefaultCamelContext.startRouteDefinitions(DefaultCamelContext.java:716)
	at org.apache.camel.impl.engine.AbstractCamelContext.doInit(AbstractCamelContext.java:2757)
	at org.apache.camel.support.service.BaseService.init(BaseService.java:83)
	at org.apache.camel.impl.engine.AbstractCamelContext.init(AbstractCamelContext.java:2475)
	at org.apache.camel.support.service.BaseService.start(BaseService.java:111)
	at org.apache.camel.impl.engine.AbstractCamelContext.start(AbstractCamelContext.java:2494)
	at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:245)
	at org.apache.camel.main.SimpleMain.doStart(SimpleMain.java:43)
	at org.apache.camel.support.service.BaseService.start(BaseService.java:119)
	at org.apache.camel.kafkaconnector.CamelSourceTask.start(CamelSourceTask.java:172)
	... 9 more
Caused by: org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: couchdb://http:couchdb:5984/local?password=xxxxxx due to: Invalid URI. Format must be of the form couchdb:http[s]://hostname[:port]/database?[options...]
	at org.apache.camel.impl.engine.AbstractCamelContext.doGetEndpoint(AbstractCamelContext.java:962)
	at org.apache.camel.impl.engine.AbstractCamelContext.getEndpoint(AbstractCamelContext.java:844)
	at org.apache.camel.support.CamelContextHelper.getMandatoryEndpoint(CamelContextHelper.java:58)
	at org.apache.camel.reifier.AbstractReifier.resolveEndpoint(AbstractReifier.java:177)
	at org.apache.camel.reifier.RouteReifier.doCreateRoute(RouteReifier.java:94)
	at org.apache.camel.reifier.RouteReifier.createRoute(RouteReifier.java:74)
	... 21 more
Caused by: java.lang.IllegalArgumentException: Invalid URI. Format must be of the form couchdb:http[s]://hostname[:port]/database?[options...]
	at org.apache.camel.component.couchdb.CouchDbEndpoint.<init>(CouchDbEndpoint.java:91)
	at org.apache.camel.component.couchdb.CouchDbComponent.createEndpoint(CouchDbComponent.java:37)
	at org.apache.camel.component.couchdb.CouchDbComponent.createEndpoint(CouchDbComponent.java:25)
	at org.apache.camel.support.DefaultComponent.createEndpoint(DefaultComponent.java:171)
	at org.apache.camel.impl.engine.AbstractCamelContext.doGetEndpoint(AbstractCamelContext.java:928)
	... 26 more

It seems the Kafka connector is building an invalid URI out of the configuration values:

couchdb://http:couchdb:5984/local?password=xxxxxx

instead of

couchdb:http://couchdb:5984/local?password=xxxxxx

The later being the valid one (apparently). Could this issue be related to https://github.com/apache/camel-kafka-connector/issues/629 ?

Anyhow, I've seen that the CouchDB source/sink connectors are no longer available on the latest 3.x versions. Is this connector deprecated? If so, why? Is there a viable alternative to it?

millenc avatar Nov 02 '22 14:11 millenc