时间精度问题

[huangfuProject]

Search before asking

• [X] I had searched in the issues and found no similar issues.

What happened

目前项目上使用 时间戳当作增量字段【startLocation】，同步dm数据，当同步2009年9月9号之前的数据的时候，项目报错：

   Unknown time unit:startLocation=xxxxxxxxxx

经查验源代码中有如下逻辑，代码位置 com.dtstack.chunjun.connector.jdbc.util.JdbcUtil#getMillis 代码如下：

    /**
     * 将边界位置时间转换成对应饿的毫秒时间
     *
     * @param startLocation 边界位置(起始/结束)
     * @return
     */
    public static long getMillis(long startLocation) {
        String timeStr = String.valueOf(startLocation);
        long millisSecond;
        if (timeStr.length() == SECOND_LENGTH) {
            millisSecond = startLocation * 1000;
        } else if (timeStr.length() == MILLIS_LENGTH) {
            millisSecond = startLocation;
        } else if (timeStr.length() == MICRO_LENGTH) {
            millisSecond = startLocation / 1000;
        } else if (timeStr.length() == NANOS_LENGTH) {
            millisSecond = startLocation / 1000000;
        } else {
            throw new IllegalArgumentException("Unknown time unit:startLocation=" + startLocation);
        }

        return millisSecond;
    }

经查验，上述代码对于时间单位的判断是基于时间戳长度来判断的

    /** 秒级时间戳的长度为10位 */
    private static final int SECOND_LENGTH = 10;
    /** 毫秒级时间戳的长度为13位 */
    private static final int MILLIS_LENGTH = 13;
    /** 微秒级时间戳的长度为16位 */
    private static final int MICRO_LENGTH = 16;
    /** 纳秒级时间戳的长度为19位 */
    private static final int NANOS_LENGTH = 19;

但是 2001-09-09 09:46:39 之前的毫秒级时间戳为 999999999999,长度为12位，导致在上述代码判断中，出现无法判断单位的情况！所以报错：

Unknown time unit:startLocation=999517080000

What you expected to happen

当我使用毫秒且时间为 2001-09-09 09:46:39 之前的数据，希望可用！

How to reproduce

rdb类型的数据，startLocation数值为2001-09-09 09:46:39之前的时间

Anything else

No response

Version

master

Are you willing to submit PR?

• [X] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[huangfuProject]

补充配置文件

{
    "job": {
        "content": [
            {
                "reader": {
                    "parameter": {
                        "password": "xxxxxxxxxxxx",
                        "startLocation": "2001-09-03 19:38:00",
                        "increColumn": "CS_TSTP",
                        "useMaxFunc": false,
                        "column": [
                            {
                                "name": "AGE",
                                "type": "NUMBER"
                            },
                            {
                                "name": "BIRTH2",
                                "type": "DATE"
                            },
                            {
                                "name": "CESHI2",
                                "type": "VARCHAR2"
                            },
                            {
                                "name": "CESHIINT",
                                "type": "INT"
                            },
                            {
                                "format": "yyyy-MM-dd HH:mm:ss",
                                "name": "CS_TSTP",
                                "type": "TIMESTAMP"
                            },
                            {
                                "name": "HIGH",
                                "type": "NUMBER"
                            },
                            {
                                "name": "ID",
                                "type": "NUMBER"
                            },
                            {
                                "name": "MOMOU",
                                "type": "VARCHAR2"
                            },
                            {
                                "name": "NAME",
                                "type": "VARCHAR2"
                            },
                            {
                                "name": "WEIGHT",
                                "type": "NUMBER"
                            }
                        ],
                        "increment": true,
                        "connection": [
                            {
                                "schema": "HUANGFU",
                                "jdbcUrl": [
                                    "jdbc:dm://xxxxxxxxx:xxxxx"
                                ],
                                "table": [
                                    "PERSON_INFO"
                                ]
                            }
                        ],
                        "where": "",
                        "polling": true,
                        "increColumnType": "TIMESTAMP",
                        "username": "HUANGFU"
                    },
                    "name": "dmreader"
                },
                "writer": {
                    "parameter": {
                        "tableFields": [
                            "AGE",
                            "BIRTH2",
                            "CESHI2",
                            "CESHIINT",
                            "CS_TSTP",
                            "HIGH",
                            "ID",
                            "MOMOU",
                            "NAME",
                            "WEIGHT"
                        ],
                        "producerSettings": {
                            "retries": "3",
                            "request.timeout.ms": "60000",
                            "batch.size": "16384",
                            "acks": "0",
                            "bootstrap.servers": "xxxx:9092,xxxx:9092,xxxxx:9092",
                            "buffer.memory": "33554432"
                        },
                        "topic": "dm_tp_w_0530"
                    },
                    "name": "kafkawriter"
                }
            }
        ],
        "setting": {
            "speed": {
                "bytes": 1024
            }
        }
    }
}

[FlechazoW]

issue 描述中的startLocaltion 不是标准的时间戳格式，如果转成正常的时间戳格式呢？

[FlechazoW]

抱歉，最近比较忙，没有及时回复。

[huangfuProject]

issue 描述中的startLocaltion 不是标准的时间触发格式，如果转成正常的时间触发格式呢？ 我不太明白您说的正常的时间触发格式，数据库的数据是存在大量的2001-09-03之前的数据的，默认毫秒的话，这个时间之前的数据全部都不满足长度的！