ElasticSearch sink does not work mapping a topic to a custom index name

[holandajunior]

Hey everyone,

I'm trying to figure out how I can map a topic name to an index using the message's timestamp field. Unfortunately, it's not working by using MessageTimestampRoute, as properly described in "Limitations" here.

However, it is a bit confusing, since this was supposed to be fixed here -- The same error that is happening with me...

Is there a proper way to achieve this topic -> index mapping?

Environment:

• kafka-connect-elasticsearch:11.1.3
• connect-transforms:1.4.1
• Confluent Platform - k8s strimzi kafka operator:0.19.0 - kafka:2.5.0

Thanks in advance!

[fidemin]

The configuration option topic.index.map existed before connect sink version 11.0.0. It was removed because ES client was just changed. I don't know the reason why it was decided to be removed.

• https://github.com/confluentinc/kafka-connect-elasticsearch/pull/470
• https://docs.confluent.io/kafka-connect-elasticsearch/current/changelog.html#version-11-0-0

[kiv06041992]

I met same problem. Deleting of topic.index.map is very strange decision

[Protoss78]

You can work around this by using this SMT: https://docs.confluent.io/platform/current/connect/transforms/regexrouter.html