dtle icon indicating copy to clipboard operation
dtle copied to clipboard

kafka full replication job column type timestamp: set default value is CURRENT_TIMESTAMP or NOW, but not output in kafka message

Open asiroliu opened this issue 4 years ago • 0 comments

Description

kafka full replication job column type timestamp: set default value is CURRENT_TIMESTAMP or NOW, but not output in kafka message

Steps to reproduce the issue

  1. create db and table
create database action_db;
use action_db

create table timestamp_columns( id int(11) not null primary key, c_timestamp timestamp DEFAULT CURRENT_TIMESTAMP)ENGINE=InnoDB DEFAULT CHARSET=utf8;
  1. create kafka job
{
  "Name": "column_types_default_full_timestamp_columns",
  "Datacenters": [
    "dc1"
  ],
  "Tasks": [
    {
      "Type": "Src",
      "Driver": "MySQL",
      "NodeId": "da1c766d-b945-9ad4-a434-458d637f4420",
      "Config": {
        "ConnectionConfig": {
          "Host": "172.100.9.2",
          "Port": 3306,
          "User": "test_src",
          "Password": "test_src"
        },
        "ReplicateDoDb": [
          {
            "TableSchema": "action_db",
            "Tables": [
              {
                "TableName": "timestamp_columns"
              }
            ]
          }
        ]
      }
    },
    {
      "Type": "Dest",
      "Driver": "Kafka",
      "NodeId": "8db2080b-2753-9ce6-a264-45d3383ef35a",
      "Config": {
        "Topic": "dtle",
        "Brokers": [
          "172.100.9.21:9092"
        ],
        "Converter": "json"
      }
    }
  ]
}
  1. insert some data
use action_db
insert into timestamp_columns values (0,'1970-01-01 08:00:01');
insert into timestamp_columns values (1,'2038-01-19 11:14:07');
insert into timestamp_columns values (2,'2015-08-12 14:57:05');
insert into timestamp_columns values (3,now());
insert into timestamp_columns values (4,CURRENT_TIMESTAMP);
insert into timestamp_columns values (5,NULL);
insert into timestamp_columns values (6,default);
  1. get message from kafka
/kafka/bin/kafka-console-consumer.sh --bootstrap-server 172.100.9.21:9092 --from-beginning --property print.key=true --topic dtle.action_db.timestamp_columns

Describe the results you received

image

Describe the results you expected

https://debezium.io/documentation/reference/1.3/connectors/mysql.html#_temporal_values image

Output of ./dtle version:**

NOMAD_VERSION=<0.11.1>
DTLE_VERSION=<9.9.9.9-master-f8a8a93>

asiroliu avatar Oct 19 '20 09:10 asiroliu