doris-flink-connector icon indicating copy to clipboard operation
doris-flink-connector copied to clipboard

Published 20 hours ago • verified publisher icon apache

Release Note 1.4.0

Open JNSimba opened this issue 1 year ago • 7 comments

Feature & Improvement

  1. Support Flink1.17 version
  2. Support Synchronize the mysql database In version 1.4.0, FlinkCDC is integrated to support MySQL's entire database synchronization. Users can use Connector to quickly access the upstream business database.
  3. Asynchronous Lookup Join was introduced in version 1.4.0, and on this basis, the logic of querying and batching was added.
  4. 1.4.0 introduced the thrift-service sdk, which blocked the Thrfit plug-in and lowered the threshold for compiling.
  5. when there is no data import, the StreamLoad connection import will not be enabled
  6. Version 1.4.0 introduces a polling mechanism, and BE nodes are replaced during each Checkpoint, avoiding the pressure of a single node acting as a Coordinator for a long time.
  7. Support reading Doris's DecimalV3/DateV2/DateTimev2/Array/JSONB type (Doris must be 2.0 or above).

Bug

  1. Fix the concurrency problem when streamload link check
  2. Fix the problem that some timeout parameters of connection doris do not take effect
  3. Fix the problem that the hidden separator does not take effect when writing
  4. Fix the problem that the query plan of reading dori is too long and the serialization fails

Thanks

Thanks to everyone who has contributed to this release: @caoliang-web @DongLiang-0 @gnehil @GoGoWen @JNSimba @legendtkl @lsy3993 @Myasuka @wolfboys

JNSimba avatar Jun 13 '23 07:06 JNSimba

大佬, Slack求助

EddyChina avatar Jun 27 '23 09:06 EddyChina

大佬, Slack求助

Can you describe the specific problem? Or contact me via WeChat, my WeChat ID is lh0809

gnehil avatar Jun 27 '23 10:06 gnehil

大佬，这个1.4.0版本，还兼容flink 1.14.x吗

DKfengyi avatar Jul 04 '23 08:07 DKfengyi

大佬，这个1.4.0版本，还兼容flink 1.14.x吗

The 1.4.0 version of the doris connector requires a minimum flink version of 1.15

gnehil avatar Jul 05 '23 02:07 gnehil

大佬，这个1.4.0版本，还兼容flink 1.14.x吗

The 1.4.0 version of the doris connector requires a minimum flink version of 1.15 了解了，多谢！

DKfengyi avatar Jul 05 '23 02:07 DKfengyi

大佬，用doris-2.0版本读取数据时候be会crash，有遇到过这样问题吗？详细信息可以看下这里： https://github.com/apache/doris-flink-connector/issues/172

liuyang185 avatar Aug 08 '23 02:08 liuyang185

1.4.0的DorisCatalog创建时

this.jdbcUrl = jdbcUrl.endsWith("/") ? jdbcUrl : jdbcUrl + "/";

这行代码是给后面拼接数据库提供便利, 但是会造成解析jdbc连接配置异常, 比如 jdbc:mysql://abc:9030?useSSL=false 在后续解析就会导致解析成 'false/'

当前master使用DorisConnectionOptions后好像没再这么干了; 不过想咨询下何时发布到下个版本?

Tyxuani avatar Sep 27 '23 02:09 Tyxuani