seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Fix][Connector-V2] Fix indefinite blocking by adjusting stopping condition in kafka split reader

Open xiaochen-zhou opened this issue 5 months ago • 3 comments

Purpose of this pull request

Fix indefinite blocking by adjusting stopping condition in split reader refer: https://github.com/apache/flink-connector-kafka/pull/100

Does this PR introduce any user-facing change?

no

How was this patch tested?

Add tests KafkaTransactionalStoppingConditionTest#testStoppingConditionWithControlMessages()

Consider partition 0, where offsets 13 and 14 are valid records and 15 is a control record. If stoppingOffset is set to 15 for partition 0and properties.max.poll.records is configured to 1, checking only partitions that return records would miss offset 15. By consistently reviewing all assigned partitions, the consumer’s position jumps control record in the subsequent poll, allowing the system to escape.

Check list

  • [ ] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
  • [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
  • [ ] If you are contributing the connector code, please check that the following files are updated:
    1. Update plugin-mapping.properties and add new connector information in it
    2. Update the pom file of seatunnel-dist
    3. Add ci label in label-scope-conf
    4. Add e2e testcase in seatunnel-e2e
    5. Update connector plugin_config

xiaochen-zhou avatar Jul 21 '25 05:07 xiaochen-zhou

Friendly ping, do you have time to take a look @Hisoka-X @Carl-Zhou-CN

xiaochen-zhou avatar Jul 21 '25 12:07 xiaochen-zhou

Done. @Hisoka-X @Carl-Zhou-CN

xiaochen-zhou avatar Aug 10 '25 15:08 xiaochen-zhou

This pull request has been automatically marked as stale because it has not had recent activity for 120 days. It will be closed in 7 days if no further activity occurs.

github-actions[bot] avatar Dec 11 '25 00:12 github-actions[bot]

This pull request has been closed because it has not had recent activity. You could reopen it if you try to continue your work, and anyone who are interested in it are encouraged to continue work on this pull request.

github-actions[bot] avatar Dec 19 '25 00:12 github-actions[bot]