support millisecond timestamps in iceberg compatibility mode

Open gmdfalk opened this issue 3 months ago • 3 comments

Purpose

This adds to https://github.com/apache/paimon/pull/4318 by allowing Paimon millisecond type to be converted to the canonical microseconds in Iceberg.

We ingest data from Kafka and other sources into Paimon and our Iceberg-based Data Lake. Kafka timestamps and some business data has timestamps with millisecond precision. This allows supporting that type of data without any type changes required in the ingestion pipeline.

Tests

IcebergConversionsTimestampTest.java

Sep 29 '25 08:09 gmdfalk

Please fix IcebergDataFieldTest.testTimestampPrecisionValidation.

Sep 30 '25 01:09 JingsongLi

Please fix IcebergDataFieldTest.testTimestampPrecisionValidation.

Done, i've also updated the docs

Sep 30 '25 10:09 gmdfalk

@JingsongLi good to merge?

Sep 30 '25 14:09 gmdfalk