flink-connector-elasticsearch
flink-connector-elasticsearch copied to clipboard
[FLINK-35287] Builder builds NetworkConfig for Elasticsearch connector 8
https://issues.apache.org/jira/browse/FLINK-35287
In FLINK-26088 we added support for ElasticSearch 8.0. It is based on Async sink API and does not use the base module flink-connector-elasticsearch-base. Regarding the config options (host, username, password, headers, ssl...), we pass all options from the builder to AsyncSink, and last to AsyncWriter. It is less flexible when we add new options and the constructors will get longer and multiple places may validate options unnecessarily. I think it's nice if we make the sink builder builds the NetworkConfig once, and pass it all the way to the writer. This is also how the base module for 6.x / 7.x is implemented.
In my recent work adding new options to the network config, this way works simpler. I'll port that work to 8.x
- builder builds
NetworkConfig
and pass that into sink and writer, instead of all specific config options - other small fixes in the builder
@mtfelisb and @reswqa mind taking a look? Thanks!
@reswqa @MartijnVisser Could you help review? Thanks