Messages not being sent back to DLQ on failure causing connector on error out

[RishabhAcodes]

We're registering Questdb with the Kafka connect via curl using the following:

curl -s -X PUT -H "Content-Type:application/json" http://localhost:8083/connectors/questdb-dietlytics/config -d '{
  "connector.class": "io.questdb.kafka.QuestDBSinkConnector",
  "topics": "dietlytics",
  "client.conf.string": "http::addr=questdb:9000;",
  "name": "questdb-dietlytics",
  "value.converter": "io.confluent.connect.avro.AvroConverter",
  "value.converter.schema.registry.url": "'"$SCHEMA_REGISTRY_URL"'",
  "value.converter.value.subject.name.strategy": "io.confluent.kafka.serializers.subject.RecordNameStrategy",
  "include.key": false,
  "key.converter": "io.confluent.connect.avro.AvroConverter",
  "key.converter.schema.registry.url": "'"$SCHEMA_REGISTRY_URL"'",
  "table": "dietlytics",
  "symbols": "event_type",
  "timestamp.field.name": "timestamp",
  "timestamp.string.format": "yyyy-MM-ddTHH:mm:ss.SSSZ",
  "value.converter.schemas.enable": true,
  "errors.tolerance": "all",
  "errors.log.enable": "true",
  "errors.deadletterqueue.topic.name": "dead-letter-example"
}'

Now, our Kafka broker has a main topic, and a DLQ topic created already.

Expected: When the producer sends an incorrectly formatted message probably incorrect timestamp in our case, Kafka connect should reject, log the error, send the message to the DLQ and continue processing as normal.

Actual When the producer sends an incorrect timestamp format, Kafka connect logs an Uncaught exception:... (QuestDB connector error here), disconnects node and shuts down altogether.

Current Fix Reset the offset manually from Kafka broker and restart the Kafka connect to restart the QuestDB connect service which then starts processing as normal.