kafka-connect-elasticsearch
kafka-connect-elasticsearch copied to clipboard
confluentinc
Error: Failed to check if index exists
Hi,
We are facing similar issue similar to https://github.com/confluentinc/kafka-connect-elasticsearch/issues/481, pasted the errors that we got and conf which we used. Could you please check this once
Logs:
ERROR WorkerSinkTask{id=elasticsearch-connector-3} Task threw an uncaught and un recoverable exception. Task is being killed and will not recover until manually restarted. Error: Failed to check if i ndex count-test exists. (org.apache.kafka.connect.runtime.WorkerSinkTask) [task-thread-elastics earch-connector-3] org.apache.kafka.connect.errors.ConnectException: Failed to check if index count-test exists. at io.confluent.connect.elasticsearch.ElasticsearchClient.callWithRetries(ElasticsearchClient.java:360) at io.confluent.connect.elasticsearch.ElasticsearchClient.indexExists(ElasticsearchClient.java:285) at io.confluent.connect.elasticsearch.ElasticsearchClient.createIndex(ElasticsearchClient.java:184) at io.confluent.connect.elasticsearch.ElasticsearchSinkTask.ensureIndexExists(ElasticsearchSinkTask.java:163) at io.confluent.connect.elasticsearch.ElasticsearchSinkTask.put(ElasticsearchSinkTask.java:90) at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:581) at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:329) at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:232) at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:201) at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:182) at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:231) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: org.apache.kafka.connect.errors.ConnectException: Failed to check if index count-test exists at io.confluent.connect.elasticsearch.RetryUtil.callWithRetries(RetryUtil.java:164) at io.confluent.connect.elasticsearch.RetryUtil.callWithRetries(RetryUtil.java:120) at io.confluent.connect.elasticsearch.ElasticsearchClient.callWithRetries(ElasticsearchClient.java:353) ... 15 more Caused by: java.net.SocketTimeoutException: 3,000 milliseconds timeout on connection http-outgoing-29 [ACTIVE] at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:789)
Conf used:
tasksMax: 3 class: io.confluent.connect.elasticsearch.ElasticsearchSinkConnector config:
connection.url: elasticsearch.es.amazonaws.com:7200 topics: count-test-topic type.name: _doc value.converter: io.confluent.connect.avro.AvroConverter value.converter.schema.registry.url: https://schema-registry:8081 value.converter.schemas.enable: true key.ignore: true behavior.on.malformed.documents: IGNORE behavior.on.null.values: IGNORE drop.invalid.message: true schema.ignore: true proxy.host: proxy.internal.com proxy.port: 6060 transforms: "dropPrefix" transforms.dropPrefix.type: "org.apache.kafka.connect.transforms.RegexRouter" transforms.dropPrefix.regex: "count-test-topic" transforms.dropPrefix.replacement: "count-test" value.converter.schema.registry.ssl.truststore.location: /opt/truststore.jks value.converter.schema.registry.ssl.truststore.password: xxxxxxx value.converter.schema.registry.ssl.keystore.location: /opt/keystore.jks value.converter.schema.registry.ssl.keystore.password: xxxxxxx value.converter.schema.registry.ssl.key.password: xxxxxxxxx
