starrocks-connector-for-apache-flink icon indicating copy to clipboard operation
starrocks-connector-for-apache-flink copied to clipboard

Published 20 hours ago • verified publisher icon StarRocks

[Feature] Support partial update

Open camilesing opened this issue 2 years ago • 8 comments

  • Background: Connector Does't support partial update will throw exception like "Fields count of test_tbl mismatch. flinkSchema[8]:k1,k2,v1,v2,v3,v4,v5,v6\n realTab[7]:k1,k2,v1,v2,v3,v4,v5"
  • Support new option sink.partial.update
  • Update README
  • Add Test Case

camilesing avatar Aug 29 '22 11:08 camilesing

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Aug 29 '22 11:08 CLAassistant

@camilesing Thanks for the contribution. Could you sign the CLA?

chaoyli avatar Sep 03 '22 01:09 chaoyli

@camilesing We have introduced a StarRocksSinkManagerV2 to optimize the memory cost for the exactly-once mode, but it can only be used for the coming StarRocks 2.4, and StarRocksSinkManager will be used for StarRocks 2.3 and other earlier version. I think partial update also need to be supported in StarRocksSinkManagerV2. Could you please rebase the newest code, and implement the logic in StarRocksSinkManagerV2?

banmoy avatar Sep 03 '22 05:09 banmoy

@chaoyli what is the CLA?

camilesing avatar Sep 06 '22 03:09 camilesing

@banmoy No problem. i'll do it when end of logical discussion with @xlfjcg

camilesing avatar Sep 06 '22 03:09 camilesing

@chaoyli what is the CLA?

@camilesing You can click the Contributor License Agreement to know more about the CLA

banmoy avatar Sep 08 '22 03:09 banmoy

@banmoy thx, i am signed

camilesing avatar Sep 09 '22 15:09 camilesing

@camilesing I have left a few comments. Hope we can merge this PR ASAP. Thanks!

banmoy avatar Sep 26 '22 09:09 banmoy