Jia Fan
Jia Fan
[Improve][Connector-V2] Support maxcompute sink writer upsert/delete action with upsert session mode
Could you refer https://github.com/dingxin-tech/maxcompute-emulator/blob/master/src/main/java/com/aliyun/odps/controller/TunnelController.java#L84 and https://github.com/apache/flink-cdc/pull/3254/files#diff-d995689e2ec048a012389e6cd18fef5e08f2d00f9115e3fb4cb22606e4b32a99 ? It should be fine with upsert tunnel. Because it already be verified in flink-cdc.
Please follow the guide to open github action on your fork repository. https://github.com/apache/seatunnel/pull/9264/checks?check_run_id=41607433608
Could you add a unit test for this? Thanks. cc @zhangshenghang
Could you add a test case for this?
Yes, please use DynamicChunkSplitter to reduce unnecessary configuration. In fact, the FixedChunkSplitter is a legacy in JDBC and is not recommended.
> query_table(just like jdbc or doris, use table_path option or similar) +1 for this, this way can better identify the metadata of the table, thereby improving read performance. > where_condition(just...
> For the case of query_table, all the parts are obtained through the system table (system.parts), and then the part data is directly read from table, similar to how doris...
> @Hisoka-X @Carl-Zhou-CN @JeremyXin So, how about we first implement the query_table approach? Or should we first implement the query-based DynamicChunkSplitter following the ClickHouse source connector's pattern? +1. Overall, `table_path`...
> @Hisoka-X In this case, do we still need to implement the query-based DynamicChunkSpillter? Yes please, query can cover many more scenarios than table_path. We can also use the ability...
we can add new provider for airflow with seatunnel. https://airflow.apache.org/docs/#active-providers