camel-kafka-connector icon indicating copy to clipboard operation
camel-kafka-connector copied to clipboard

AMQP Sink Connector very slowly

Open Alexander-ac opened this issue 2 years ago • 0 comments

Hi all, we currently use the following packages:

amqp-kafka-connector-0.11.5 and apache-qpid-jms-0.56

The sink connector (over amqp 1.0) itself works, but extremely slowly. What is immediately noticeable in the log is that the sink connector reestablishes the connection with the ActiveMQ broker after each message is delivered!!!!

for example:

[2023-03-06 20:09:48,222] INFO [AMQPSinkConnector|task-0] Connection ID:a7733de7-20c9-4a09-9314-98aab25f976f:2 connected to server: amqps://192.168.1.12:5671 (org.apache.qpid.jms.JmsConnection:1353) [2023-03-06 20:09:48,572] INFO [AMQPSinkConnector|task-0] Connection ID:c5780863-215f-4539-beb6-b1ef70b3f759:3 connected to server: amqps://192.168.1.12:5671 (org.apache.qpid.jms.JmsConnection:1353) [2023-03-06 20:09:48,759] INFO [AMQPSinkConnector|task-0] Connection ID:36c069ae-00b6-4719-9362-4bd7021ae75d:4 connected to server: amqps://192.168.1.12:5671 (org.apache.qpid.jms.JmsConnection:1353)

Here I am sending three messages and we see three connection structures. This slows down the spread of the messages.

Anyone have similar experiences?

Here is the property file of the sink connector:

name=AMQPSinkConnector tasks.max=1

connector.class=org.apache.camel.kafkaconnector.amqp.CamelAmqpSinkConnector

camel.component.amqp.testConnectionOnStartup=true camel.component.amqp.includeAmqpAnnotations=false camel.component.amqp.connectionFactory.remoteURI=amqps://192.168.1.12:5671 camel.component.amqp.connectionFactory=#class:org.apache.qpid.jms.JmsConnectionFactory camel.component.amqp.username=admin camel.component.amqp.password=admin camel.sink.path.destinationType=queue camel.sink.path.destinationName=Output.SinkConnector_2

topics=TEST_TOPIC

key.converter=org.apache.kafka.connect.storage.StringConverter value.converter=org.apache.kafka.connect.storage.StringConverter

I am grateful for every tip! Thanks in advance

Alex

Alexander-ac avatar Mar 06 '23 19:03 Alexander-ac