seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

The database is the field where oracle partition_cloumn (id) is NUMBER; Size, scale 0, restapi submit-job error

Open sandyustc opened this issue 1 year ago • 6 comments

Search before asking

  • [X] I had searched in the issues and found no similar issues.

What happened

{ "status": "fail", "message": "ErrorCode:[API-01], ErrorDescription:[Configuration item validate failed] - PluginName: jdbc, PluginType: source, Message: org.apache.seatunnel.connectors.seatunnel.jdbc.exception.JdbcConnectorException: ErrorCode:[COMMON-06], ErrorDescription:[Illegal argument] - The current field is DecimalType containing decimals: 18 Unable to support" }

SeaTunnel Version

2.3.3

SeaTunnel Config

{
	"env": {
		"parallelism": 4,
		"job.mode": "batch",
		"checkpoint.interval": 10000
	},
	"source": [{
		    "plugin_name": "jdbc",
			"url": "jdbc:oracle:thin:@xxx.xxx.11.181:1521:orcl",
			"driver": "oracle.jdbc.OracleDriver",
			"connection_check_timeout_sec": 100,
			"user": "xxx",
			"password": "xxx",
			"query": "select * from  table1",
			"partition_column": "id",
			"split.size": 10000
		
	}],
    "transform": [{
        "plugin_name": "Copy",
        "source_table_name": "xxxxx",
         "result_talbe_name": "xxxx2",
          "fields": {
               "id" : "xxxxx"
          }
         
    }

    ],
		"sink": [{
		"plugin_name": "Jdbc",
			"url": "jdbc:postgresql://xxx.xxx.11.xxx:5432/ptest",
			"driver": "org.postgresql.Driver",
			"user": "test",
			"password": "test",
			"generate_sink_sql": true,
			"database": "ptest",
			"table": "public.xxxx2",
			"schema_save_mode": "CREATE_SCHEMA_WHEN_NOT_EXIST",
			"data_save_mode": "APPEND_DATA",
			"primary_keys": ["id"]

		
	}]
}

Running Command

{
	"env": {
		"parallelism": 4,
		"job.mode": "batch",
		"checkpoint.interval": 10000
	},
	"source": [{
		    "plugin_name": "jdbc",
			"url": "jdbc:oracle:thin:@xxx.xxx.11.181:1521:orcl",
			"driver": "oracle.jdbc.OracleDriver",
			"connection_check_timeout_sec": 100,
			"user": "xxx",
			"password": "xxx",
			"query": "select * from  table1",
			"partition_column": "id",
			"split.size": 10000
		
	}],
    "transform": [{
        "plugin_name": "Copy",
        "source_table_name": "xxxxx",
         "result_talbe_name": "xxxx2",
          "fields": {
               "id" : "xxxxx"
          }
         
    }

    ],
		"sink": [{
		"plugin_name": "Jdbc",
			"url": "jdbc:postgresql://xxx.xxx.11.xxx:5432/ptest",
			"driver": "org.postgresql.Driver",
			"user": "test",
			"password": "test",
			"generate_sink_sql": true,
			"database": "ptest",
			"table": "public.xxxx2",
			"schema_save_mode": "CREATE_SCHEMA_WHEN_NOT_EXIST",
			"data_save_mode": "APPEND_DATA",
			"primary_keys": ["id"]

		
	}]
}

Error Exception

{
    "status": "fail",
    "message": "ErrorCode:[API-01], ErrorDescription:[Configuration item validate failed] - PluginName: jdbc, PluginType: source, Message: org.apache.seatunnel.connectors.seatunnel.jdbc.exception.JdbcConnectorException: ErrorCode:[COMMON-06], ErrorDescription:[Illegal argument] - The current field is DecimalType containing decimals: 18 Unable to support"
}

Zeta or Flink or Spark Version

Zeta

Java or Scala Version

java 1.8

Screenshots

No response

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

sandyustc avatar Mar 29 '24 10:03 sandyustc

What is the id field type for table1 ? is number(18,0) ?

hawk9821 avatar Apr 02 '24 10:04 hawk9821

What is the id field type for table1 ? is number(18,0) ? @hawk9821 thanks The ID field of the table is of type number(0,0)

sandyustc avatar Apr 02 '24 11:04 sandyustc

Problems with oracle data type conversion , A conversion logic error caused the oracle number type to be converted to Decimal(38,18). The expectation should be Long

hawk9821 avatar Apr 03 '24 01:04 hawk9821

Problems with oracle data type conversion , A conversion logic error caused the oracle number type to be converted to Decimal(38,18). The expectation should be Long @hawk9821 thanks

Oracle table field ID number type size and precision set to 0, should not be converted to decimal(38,18), why is it converted to this, what special requirements, if this is limited, that the column name for split data ,it can not be processed("partition_column": "id")

sandyustc avatar Apr 03 '24 07:04 sandyustc

image image The oracle number data type can store float data without specifying the precision, so seatunnel is treated as float and converted to Decimal(38,18). Change table field ID data type to number(x,0)

hawk9821 avatar Apr 07 '24 01:04 hawk9821

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] avatar May 08 '24 00:05 github-actions[bot]