seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Bug] [Connector-Iceberg] Fix Time Zone Issue for Iceberg Timestamp Type

Open yzeng1618 opened this issue 6 months ago • 0 comments

Search before asking

  • [x] I had searched in the issues and found no similar issues.

What happened

During the synchronization process from MySQL to Iceberg, the synchronization is shown as successful, but the timestamp type time is found to differ from the source by 8 hours.

SeaTunnel Version

2.3.9

SeaTunnel Config

{
  env {
    execution.parallelism = 1
    job.mode = "BATCH"
    job.name = "seatunnel_batch_job"
  }
  # source配置
  source {
    JDBC {
      url = "xxxx"
      driver = "xxxxx"
      user = "xxxx"
      password = "xxxx"
      query = "xxxxx"
      split.size = 5000
      fetch_size = 2000      
    }
  }
  # sink配置
  sink {
    Iceberg {
      catalog_name= "default"
      iceberg.catalog.config={
        "type"= "xxxx"
        "uri" = "xxxx"
        "warehouse"= "xxxx"
      }
      namespace= "xxxxx"
      table= "xxxxx"
      iceberg.table.write-props={
        write.format.default="xxxx"
        write.target-file-size-bytes=536870912
        write.parquet.compression-codec="xxxx"
      }
      iceberg.hadoop-conf-path = "xxxxx"
      iceberg.table.upsert-mode-enabled = false
      iceberg.table.schema-evolution-enabled=true
      case_sensitive=true      
    }
  }
}

Running Command

./bin/start-seatunnel-flink-15-connector-v2.sh --config mysql2iceberg.conf --deploy-mode run-application --target yarn-application --name mysql2iceberg

Error Exception

The value of the datetime type at the MySQL source end is 2020-06-08 15:21:15, and after successful synchronization, the value of the corresponding field is 2020-06-08 23:21:15.000.

Zeta or Flink or Spark Version

flink 1.16

Java or Scala Version

java 8

Screenshots

No response

Are you willing to submit PR?

  • [x] Yes I am willing to submit a PR!

Code of Conduct

yzeng1618 avatar Jun 18 '25 07:06 yzeng1618