[Bug] [CDCSOURCE] DATE column of mysql source is changed to '1970-01-01' when sinking to starrocks
Search before asking
- [X] I had searched in the issues and found no similar issues.
What happened
flink task: EXECUTE CDCSOURCE jobname WITH ( 'connector' = 'mysql-cdc', 'hostname' = 'xxxxxxxxxxx', 'port' = '3306', 'username' = 'xxxxx', 'password' = 'xxxx', 'checkpoint' = '3000', 'scan.startup.mode' = 'initial', 'parallelism' = '1', 'source.server-time-zone' = 'Africa/Nairobi', 'table-name' = 'tb.repay_order', 'sink.connector' = 'starrocks', 'sink.jdbc-url' = 'jdbc:mysql://xxxxx:9030', 'sink.load-url' = 'xxxx:8030', 'sink.username' = 'flink', 'sink.password' = 'xxxx@xxxx', 'sink.sink.db' = 'kenya_biz_olap', 'sink.table.prefix' = 'ods_tb_', 'sink.table.lower' = 'true', 'sink.database-name' = '${schemaName}', 'sink.table-name' = '${tableName}', 'sink.sink.properties.format' = 'json', 'sink.sink.properties.strip_outer_array' = 'true', 'sink.sink.max-retries' = '10', 'sink.sink.buffer-flush.interval-ms' = '15000', 'sink.sink.parallelism' = '1' )
The source data of column 'end_date' (type: DATE) is '2022-10-18'
but after sinking to starrocks, it was changed to '1970-01-01'.
The data is correct when using flink tool sql-client.sh to synchronize table.
What you expected to happen
the column of type DATA should be same with source.
How to reproduce
The enviroments: Flink 1.15.3 flink-connector-starrocks-1.2.5_flink-1.15.jar flink-sql-connector-mysql-cdc-2.3.0.jar dlink-client-1.15-0.7.1.jar dlink-client-base-0.7.1.jar dlink-common-0.7.1.jar
Table of starrocks:
CREATE DATABASE IF NOT EXISTS lending;
CREATE TABLE IF NOT EXISTS lending.table1 (
id INT(11) NOT NULL COMMENT "",
test_date DATE NOT NULL"
) ENGINE=olap
PRIMARY KEY(id)
DISTRIBUTED BY HASH(id) BUCKETS 1
PROPERTIES (
"replication_num" = "3"
);
Anything else
No response
Version
dev
Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
Check the date format in the source database (MySQL CDC) and the target database (Starrocks) and ensure that they are the same. If they are different, try to change the date format in the sink database to match the source format.
Finally I found that with Flink: 1.15.3 JDK: 11, Date column is incorrect (1970-01-01), with Flink: 1.14.5 JDK: 1.8, Date column is correct!