Out of ordering of requests to opensearch

[Ramkumar-byte]

Hi Team,

We are seeing an out of order problem for the requests to opensearch when using index.write.method=upsert.

However we don't see this issue when we set index.write.method=insert.

Below is our connector config

connector.class=io.aiven.kafka.connect.opensearch.OpensearchSinkConnector behavior.on.null.values=delete transforms.extractKey.field=id index.write.method=insert connection.password= tasks.max=10 topics=dev.task batch.size=10000 transforms=extractKey key.ignore=false max.in.flight.requests=5 key.ignore.id.strategy=none transforms.extractKey.type=org.apache.kafka.connect.transforms.ExtractField$Key schema.ignore=true key.converter.schemas.enable=false flush.timeout.ms=300000 value.converter.schemas.enable=false value.converter=org.apache.kafka.connect.json.JsonConverter connection.url=... key.converter=org.apache.kafka.connect.json.JsonConverter behavior.on.version.conflict=warn schemas.enable=false

[qlecorre]

Hey @Ramkumar-byte, thanks for reporting this issue with the sink connector. Is it still happening for you?

[nj4x]

FYI: We've experienced same issue. Apparently it can be fixed by setting max.in.flight.requests=1