[Bug] [mysql-cdc] When mysql-cdc retrieves the partitioning column, if there is a composite index, it will retrieve fields other than the primary key and index columns as the partitioning column, leading to slow SQL · Issue #9949 · apache/seatunnel

[huomengbo]

Search before asking

• [x] I had searched in the issues and found no similar issues.

What happened

同步历史数据的时候，未配置snapshotSplitColumn，会自动获取到coin_type字段作为切分列，因为数据量比较大，没有走索引，导致慢sql

表结构 CREATE TABLE user ( addr_id bigint NOT NULL AUTO_INCREMENT COMMENT '地址', user_id bigint NOT NULL DEFAULT '0' COMMENT '用户id', address varchar(255) CHARACTER SET utf8mb3 COLLATE utf8mb3_general_ci NOT NULL COMMENT '地址2', type int NOT NULL DEFAULT '3' COMMENT '类型', created_date timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间', modify_date timestamp NULL DEFAULT NULL COMMENT '修改时间', PRIMARY KEY (addr_id), UNIQUE KEY address (address,coin_type) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3

原因：在自动选择的时候，获取到了唯一索引，但是int的优先级比bigint和string的优先级高，导致选择了coin_type，但是联合索引，不指定第一个字段，第二个字段是无法使用索引的，下面源码地方，应该取消for循环，只能选择第一个字段作为切分列

源码：AbstractJdbcSourceChunkSplitter

SeaTunnel Version

2.3.10

SeaTunnel Config

-

Running Command

-

Error Exception

扫描数据库慢sql

Zeta or Flink or Spark Version

No response

Java or Scala Version

jdk8

Screenshots

No response

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [x] I agree to follow this project's Code of Conduct

[LeonYoah]

Can't we specify partition_column

[huomengbo]

Can't we specify partition_column

可以，如果我表太多，几千张表，没法一个一个配置吧，想用自动寻找切分列，这里就不行。这里就是有bug，非索引列不应该出现。

[we1she2]

2.3.12 都还没解决呢。。

[dybyte]

Hi @zhangshenghang . Could you assign it to me? Thanks!

[zhangshenghang]

Hi @zhangshenghang . Could you assign it to me? Thanks!

thanks @dybyte