从MYSQL同步数据到CLICKHOUSE,速度太慢

[sid659]

Search before asking

• [ ] I had searched in the issues and found no similar question.

• [ ] I had googled my question but i didn't get any help.

• [ ] I had read the documentation: ChunJun doc but it didn't help me.

Description

从MYSQL同步数据到CLICKHOUSE,速度太慢。 版本是2022年11月下的，现在查不到版本号了。MYSQL单表1000多万的数据，同步到ck近30小时。请问要怎么优化速度？ json: { "job": { "content": [ { "reader": { "name": "mysqlreader", "parameter": { "column": [ ... ], "customSql": "xxxxxxxxx", "fetchSize": 1024, "where": "GID > '110d4fb092dd44e68d0df3b971e86a95'", "username": "xxxx", "password": "xxxx", "connection": [ { "jdbcUrl": [ "jdbc:mysql://xxx.xxx.xxxx.xxxx:3306/xxx?serverTimezone=Asia/Shanghai&useUnicode=true&characterEncoding=UTF-8&useSSL=false" ], "table": [ "xxxxxx" ] } ] } }, "writer": { "name": "clickhousewriter", "parameter": { "username": "default", "password": "xxxxxx", "connection": [ { "jdbcUrl": "jdbc:clickhouse://xxx.xxx.xxx.xxx:8123/xxx", "table": [ "xxxxx" ] } ], "writeMode" : "insert", "encoding" : "utf-8", "column": [ ... ] } } } ], "setting": { "speed": { "channel": 1, "bytes": 0 }, "restore": { "isResore": false }, "errorLimit" : { "record": 1 }, "log" : { "isLogger": true, "level" : "debug" } } } }

Code of Conduct

• [X] I agree to follow this project's Code of Conduct