seatunnel
seatunnel copied to clipboard
[Feature][Kafka] Support multi-table source read
Purpose of this pull request
Support for kafka multiple data sources
Does this PR introduce any user-facing change?
The same as before
How was this patch tested?
e2e Adds multi-source tests including TOPIC Regex matching
Check list
- [ ] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
- [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
- [ ] If you are contributing the connector code, please check that the following files are updated:
- Update change log that in connector document. For more details you can refer to connector-v2
- Update plugin-mapping.properties and add new connector information in it
- Update the pom file of seatunnel-dist
- [ ] Update the
release-note
.
issues: https://github.com/apache/seatunnel/issues/5677
PTAL: @hailin0 @Hisoka-X
please resolve conflict
PTAL:@Hisoka-X @hailin0 thanks !
The topic parameter itself can support the reading of multiple topics. What is the difference?
The topic parameter itself can support the reading of multiple topics. What is the difference?
For example, Topic1 can be consumed from scratch,topic 2 is consumed from the latest offset,topic 3 uses canal_json to parse messages... And so on
@ruanwenjun @TyrantLucifer @hailin0 @Hisoka-X @liugddx @Carl-Zhou-CN PTAL.
PTAL: @Hisoka-X @hailin0
PTAL: @Hisoka-X @hailin0
PTAL: @Hisoka-X @hailin0