alibaba
增量同步写入KAKFA集群，kafka分区leader分布不同节点会同步报错
mongoshake version: 2.6.4/2.8.2 均测试过 mongodb version : 3.4/6.0 均测试过 Kafka version : 3.2.1 (Commit:b172a0a94f4ebb9f)
异常场景： topic状况： Topic: evc-hkg-testing-db TopicId: TBTYeXV9RUCH7vI7UISOCw PartitionCount: 8 ReplicationFactor: 3 Configs: Topic: evc-hkg-testing-db Partition: 0 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3 Topic: evc-hkg-testing-db Partition: 1 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1 Topic: evc-hkg-testing-db Partition: 2 Leader: 3 Replicas: 3,1,2 Isr: 3,1,2 Topic: evc-hkg-testing-db Partition: 3 Leader: 1 Replicas: 1,3,2 Isr: 1,3,2 Topic: evc-hkg-testing-db Partition: 4 Leader: 2 Replicas: 2,1,3 Isr: 2,1,3 Topic: evc-hkg-testing-db Partition: 5 Leader: 3 Replicas: 3,2,1 Isr: 3,2,1 Topic: evc-hkg-testing-db Partition: 6 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3 Topic: evc-hkg-testing-db Partition: 7 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1
mongoshake关键配置： tunnel.kafka.partition_number = 8 tunnel.address = [email protected]:9092,inf-xxx.com:9092,inf-xxx.com:9092 tunnel = kafka tunnel.message = json incr_sync.worker = 8 incr_sync.tunnel.write_thread = 32
错误日志： mongoshake.log.1:[2025/05/15 16:13:26 CST] [EROR] KafkaWriter[[email protected]:9092,inf-xxx.com:9092,inf-xxx.com:9092] with partiti onId[1] send [[email protected]:9092,inf-xxx.com:9092,inf-xxx.com:9092] with type[json] error[kafka server: Tried to send a message to a replica that is not the leader for some partition. Your metadata is out of date.]
正常同步场景： ./bin/kafka-topics.sh --create --bootstrap-server inf-ali-hkg-staging-kafka01.everonet.com:9092 --topic evc-hkg-testing-db --replica-assignment 1:2,1:2
Topic: evc-hkg-testing-db TopicId: v3ykne0MSg6gcLh-GNuVjQ PartitionCount: 2 ReplicationFactor: 2 Configs: Topic: evc-hkg-testing-db Partition: 0 Leader: 1 Replicas: 1,2 Isr: 1,2 Topic: evc-hkg-testing-db Partition: 1 Leader: 1 Replicas: 1,2 Isr: 1,2
tunnel.kafka.partition_number = 2
抱歉之前看到了本issue，但是由于暂时没有精力维护kafka通道，所以没有及时回复，还望理解。
我会在优先解决direct通道的几个问题后再来看此问题，也希望其他使用kafka通道且有能力的开发者来共建此issue的处理。
