seatunnel
seatunnel copied to clipboard
[Bug] [sqlserver cdc] Caused by: java.lang.UnsupportedOperationException: Don't support SqlSever type 'timestamp' yet, jdbcType:'-2'.
Search before asking
- [X] I had searched in the issues and found no similar issues.
What happened
sqlserver同步数据到doris报错不支持时间类型, 无法同步含有时间字段的数据表, 但是source 的表时间字段是datetime
SeaTunnel Version
2.3.3
SeaTunnel Config
env {
# You can set engine configuration here
execution.parallelism = 1
job.mode = "STREAMING"
execution.checkpoint.interval = 10000
}
source {
SqlServer-CDC {
result_table_name = "source_p_Project"
username = "**"
password = "***"
database-names = ["dotnet"]
table-names = ["dotnet.dbo.p_Project"]
base-url="jdbc:sqlserver://10.129.*.***:1433;databaseName=dotnet"
}
}
transform {
sql {
source_table_name = "source_p_Project"
query = """ select *, now() as load_time from source_p_Project """
result_table_name = "trans_p_Project"
}
}
sink {
Doris{
source_table_name = "trans_p_Project"
fenodes = "10.129.37.206:8030"
username = "**"
password = "***"
table.identifier = "tmp.mys_p_Project_tmp"
sink.enable-2pc = true
sink.enable-delete = true
sink.label-prefix = "p_Projectg_202401231146"
doris.config = {
format = "json"
read_json_by_line = "true"
}
}
}
Running Command
./bin/seatunnel.sh --config ./config/sqlserver2doris_1.conf
Error Exception
Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:191)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: java.lang.UnsupportedOperationException: Don't support SqlSever type 'timestamp' yet, jdbcType:'-2'.
at org.apache.seatunnel.connectors.seatunnel.cdc.sqlserver.source.utils.SqlServerTypeUtils.convertFromColumn(SqlServerTypeUtils.java:74)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:550)
at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:517)
at org.apache.seatunnel.connectors.seatunnel.cdc.sqlserver.source.utils.SqlServerTypeUtils.convertFromTable(SqlServerTypeUtils.java:88)
at org.apache.seatunnel.connectors.seatunnel.cdc.sqlserver.source.source.SqlServerIncrementalSource.createDebeziumDeserializationSchema(SqlServerIncrementalSource.java:125)
at org.apache.seatunnel.connectors.cdc.base.source.IncrementalSource.prepare(IncrementalSource.java:119)
at org.apache.seatunnel.engine.core.parse.JobConfigParser.parseSource(JobConfigParser.java:85)
at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSource(MultipleTableJobConfigParser.java:317)
at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:179)
at org.apache.seatunnel.engine.core.job.AbstractJobEnvironment.getLogicalDag(AbstractJobEnvironment.java:109)
at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.execute(JobExecutionEnvironment.java:73)
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:143)
... 2 more
Zeta or Flink or Spark Version
zeta
Java or Scala Version
No response
Screenshots
No response
Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
It has been fixed in #5668.
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.