seatunnel
seatunnel copied to clipboard
[Bug][Improve][Connector-v2][ElasticsearchSource] Fix behavior when source empty,Support SourceConfig.SOURCE field empty.
[Improve][Connector-v2][ElasticsearchSource] Support SourceConfig.SOURCE field empty.
Purpose of this pull request
Does this PR introduce any user-facing change?
How was this patch tested?
Check list
- [x] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
- [x] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
- [x] If you are contributing the connector code, please check that the following files are updated:
- Update change log that in connector document. For more details you can refer to connector-v2
- Update plugin-mapping.properties and add new connector information in it
- Update the pom file of seatunnel-dist
- [x] Update the
release-note
.
@iture123 @TyrantLucifer @Hisoka-X PTAL,thanks.
I think we should add a e2e test case in https://github.com/apache/seatunnel/blob/045d3e3494eac67cef555c190ecb68638d497349/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-elasticsearch-e2e/src/test/java/org/apache/seatunnel/e2e/connector/elasticsearch/ElasticsearchIT.java#L68
@iture123 @TyrantLucifer @Hisoka-X PTAL,thanks.
Hi @CosmosNi , thanks for add test! Could you retry the test case after merge from dev?
@Hisoka-X I don't understand. Can you show me how to operate it?
merge code from upstream
@Hisoka-X already merge code from upstream,PTAL,thanks.
The conflict has been resolved,and already add test case ,@iture123 @TyrantLucifer @Hisoka-X PTAL,thanks.
I think
schema
option is excess for ES Source, So, can we commentschema
is deprecated in code and document? @EricJoy2048 Yes, in fact, I think source is sufficient for es
Thanks @CosmosNi for update, could you follow the suggestion of @EricJoy2048 to mark schema config in ES source as deprecated? Both in doc and code.
Thanks @CosmosNi for update, could you follow the suggestion of @EricJoy2048 to mark schema config in ES source as deprecated? Both in doc and code.
@Hisoka-X mark schema config in ES source as deprecated already
https://github.com/apache/seatunnel/pull/6425/files#diff-8eef53f88ae098e1477f67d8053f946d3267578ec8e68fd472d22c298864f876R60
This line should be deleted. Also please tell user when not config source
, the behavior what is.
@Hisoka-X I have a problem,There are no array types in es,so when source include an array type,The engine will not be able to fight for the job.It is not up to the user to specify the array type to be used by adding parameters
@Hisoka-X I have a problem,There are no array types in es,so when source include an array type,The engine will not be able to fight for the job.It is not up to the user to specify the array type to be used by adding parameters
Yes, this is an optional way.
@Hisoka-X i add a label (array_column) to handle es array type,PTAL,thanks.
@Hisoka-X @hailin0 @EricJoy2048 PTAL,thanks.
@Hisoka-X sorry,I accidentally removed label.