chunjun icon indicating copy to clipboard operation
chunjun copied to clipboard

[Bug] [chunjun-core][enums] NUMBER_TYPE类型缺少了postgreSQL的INT2,INT4,INT8的数据类型

Open weijunzhe opened this issue 2 years ago • 1 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

What happened

我在使用postgresql增量同步时发现了下面的int4类型报错。

2023-06-27 16:34:35.549 [main] INFO  com.dtstack.chunjun.util.PluginUtil  - ChunJun executionMode: local
2023-06-27 16:34:35.549 [main] INFO  com.dtstack.chunjun.util.PluginUtil  - ChunJun reset pipeline.jars: [/usr/local/chunjun/chunjun-dist/connector/postgresql/chunjun-connector-postgresql.jar, /usr/local/chunjun/chunjun-dist/chunjun-core.jar, /usr/local/chunjun/chunjun-dist/metrics/prometheus/chunjun-metrics-prometheus.jar]
2023-06-27 16:34:35.552 [main] INFO  com.dtstack.chunjun.classloader.ClassLoaderManager  - jarUrl:file:/usr/local/chunjun/chunjun-dist/chunjun-core.jar_file:/usr/local/chunjun/chunjun-dist/connector/postgresql/chunjun-connector-postgresql.jar_file:/usr/local/chunjun/chunjun-dist/metrics/prometheus/chunjun-metrics-prometheus.jar create ClassLoad successful...
Exception in thread "main" com.dtstack.chunjun.throwable.ChunJunRuntimeException: Unsupported columnType [int4], columnName [onetb]
	at com.dtstack.chunjun.connector.jdbc.dialect.JdbcDialect.initKeyUtil(JdbcDialect.java:396)
	at com.dtstack.chunjun.connector.jdbc.source.JdbcSourceFactory.initIncrementConfig(JdbcSourceFactory.java:279)
	at com.dtstack.chunjun.connector.jdbc.source.JdbcSourceFactory.createSource(JdbcSourceFactory.java:115)
	at com.dtstack.chunjun.Main.exeSyncJob(Main.java:184)
	at com.dtstack.chunjun.Main.main(Main.java:118)
	at com.dtstack.chunjun.client.local.LocalClusterClientHelper.submit(LocalClusterClientHelper.java:35)
	at com.dtstack.chunjun.client.Launcher.main(Launcher.java:119)

What you expected to happen

通过报错位置排查到是枚举类com.dtstack.chunjun.enums.ColumnType里面的NUMBER_TYPE集合没有PostgreSql的INT2,INT4,INT8类型。因为在该类的47行有新加的注释// for pg INT2,INT4,INT8

image 加上后应该就没问题了。 image

How to reproduce

{
    "job": {
      "content": [
        {
          "reader": {
            "name": "postgresqlreader",
            "parameter": {
              "column": [
                {
                  "name": "onetb",
                  "type": "int"
                },
                {
                    "name": "twotb",
                    "type": "int"
                },
                {
                    "name": "colltb",
                    "type": "int"
                }
              ],
              "polling": true,
              "pollingInterval": 5000,
              "increColumn": "onetb",
              "useMaxFunc": true,
              "username": "postgres",
              "password": "123456",
              "connection": [
                {
                  "jdbcUrl": [
                    "jdbc:postgresql://127.0.0.1:5432/test?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&serverTimezone=Asia/Shanghai"
                  ],
                  "table": [
                    "test"
                  ]
                }
              ]
            }
          },
          "writer": {
            "name": "postgresqlwriter",
            "parameter": {
              "username": "postgres",
              "password": "123456",
              "connection": [
                {
                  "jdbcUrl": "jdbc:postgresql://127.0.0.1:5432/test?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true&serverTimezone=Asia/Shanghai",
                  "table": [
                    "test_copy1"
                  ]
                }
              ],
              "writeMode": "insert",
              "column": [
                {
                  "name": "onetb",
                  "type": "int"
                },
                {
                    "name": "twotb",
                    "type": "int"
                },
                {
                    "name": "colltb",
                    "type": "int"
                }

              ]
            }
          }
        }
      ],
      "setting": {
        "speed": {
          "channel": 1,
          "bytes": 0
        }
      }
    }
}

Anything else

No response

Version

1.12_release

Are you willing to submit PR?

  • [X] Yes I am willing to submit a PR!

Code of Conduct

weijunzhe avatar Jun 27 '23 09:06 weijunzhe

it's same for ck

sailyfirm avatar Mar 26 '24 03:03 sailyfirm