seatunnel
seatunnel copied to clipboard
[Bug] [Connector-Iceberg] Fix Time Zone Issue for Iceberg Timestamp Type
Search before asking
- [x] I had searched in the issues and found no similar issues.
What happened
During the synchronization process from MySQL to Iceberg, the synchronization is shown as successful, but the timestamp type time is found to differ from the source by 8 hours.
SeaTunnel Version
2.3.9
SeaTunnel Config
{
env {
execution.parallelism = 1
job.mode = "BATCH"
job.name = "seatunnel_batch_job"
}
# source配置
source {
JDBC {
url = "xxxx"
driver = "xxxxx"
user = "xxxx"
password = "xxxx"
query = "xxxxx"
split.size = 5000
fetch_size = 2000
}
}
# sink配置
sink {
Iceberg {
catalog_name= "default"
iceberg.catalog.config={
"type"= "xxxx"
"uri" = "xxxx"
"warehouse"= "xxxx"
}
namespace= "xxxxx"
table= "xxxxx"
iceberg.table.write-props={
write.format.default="xxxx"
write.target-file-size-bytes=536870912
write.parquet.compression-codec="xxxx"
}
iceberg.hadoop-conf-path = "xxxxx"
iceberg.table.upsert-mode-enabled = false
iceberg.table.schema-evolution-enabled=true
case_sensitive=true
}
}
}
Running Command
./bin/start-seatunnel-flink-15-connector-v2.sh --config mysql2iceberg.conf --deploy-mode run-application --target yarn-application --name mysql2iceberg
Error Exception
The value of the datetime type at the MySQL source end is 2020-06-08 15:21:15, and after successful synchronization, the value of the corresponding field is 2020-06-08 23:21:15.000.
Zeta or Flink or Spark Version
flink 1.16
Java or Scala Version
java 8
Screenshots
No response
Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
Code of Conduct
- [x] I agree to follow this project's Code of Conduct