dinky icon indicating copy to clipboard operation
dinky copied to clipboard

[Bug] CDCSOURCE mysql cdc date类型写入pg, 数值有误, 全部为1970-01-01

Open cooltnt opened this issue 2 years ago • 1 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

What happened

目前所用版本 dinky 0.7.1 flink 1.16.0 flinkcdc 2.3.0

  1. cdcsource方式将含date类型字段的表, 同步到postgresql , 首次全量初始化完成后, 发现目标表对应字段值均为1970-01-01, 但是修改下源表数据, 该记录date字段值变正确.
  2. 用sink方式单表同步, 不存在此问题.

发现issues中有相关已关闭问题 https://github.com/DataLinkDC/dinky/pull/799 https://github.com/DataLinkDC/dinky/issues/821

What you expected to happen

希望能正确同步date类型字段

How to reproduce

单表sink方式同步

Anything else

No response

Version

0.7.0

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

cooltnt avatar Feb 02 '23 08:02 cooltnt

Dinky 0.7.1, Flink 1.16.0, MySQL 8.0.29, CDC connectors 2.3.0, Doris 1.2.1 环境下,在 CDCSOURCE 整库同步 mysql 到 doris 时同样遇到此问题!

leeoo avatar Feb 20 '23 02:02 leeoo

Dinky 0.7.1, Flink 1.16.0, MySQL 8.0.29, CDC connectors 2.3.0, Doris 1.2.1 环境下,在 CDCSOURCE 整库同步 mysql 到 doris 时同样遇到此问题!

image 我无法复现该问题

aiwenmo avatar Feb 28 '23 12:02 aiwenmo

同样遇到此问题 环境: Dinky 0.7.3 Doris 1.1 Flink 1.16.0 & Flink 1.17.0 在尝试修改源码解决中

cygnusdark avatar May 17 '23 09:05 cygnusdark

同样遇到此问题,用的 datetime 类型字段 环境: Dinky: v0.7.3 数据库: MySQL 5.7 Kafka: v2.4.1 Flink: 1.16.0 & 1.17.0

adu-shzz avatar May 18 '23 02:05 adu-shzz

pg整库到hudi 0.12.2
数据类型为date hudi里均为1970-01-01

javaht avatar Sep 19 '23 07:09 javaht

com.dlink.cdc.AbstractSinkBuilder.getLogicalType

case TIMESTAMP: return new TimestampType(column.getLength());

aiwenmo avatar Nov 20 '23 15:11 aiwenmo

  • [X] #2557

aiwenmo avatar Nov 20 '23 15:11 aiwenmo