trino
trino copied to clipboard
trinodb
Support timestamp type in Iceberg migrate procedure
Description
Fixes https://github.com/trinodb/trino/issues/17006
Hive: It occurs that Hive takes the timestamp and adjust the timestamp to get UTC value according to the writer's TZ. It saves the timestamp in UTC and adds also the writer TZ info to the footer. During read, it reads the timestamp in UTC and adjust it according to the writer's TZ.
Trino uses the following property hive.parquet.time-zone for setting Parquet reader's and Parquet writer's timezone when dealing with Hive.
Iceberg: Trino always sets the UTC timezone for Parquet reader's and Parquet writer's timezone. In this way, it always return UTC values.
Change: In iceberg during reading Parquet file, when footer contains TZ, the TZ value is used and the timestamp are adjusted according to the TZ value.
https://github.com/trinodb/trino/issues/17785
Release notes
(x) Release notes are required, with the following suggested text:
# Iceberg
* Support `timestamp(3)` type in `migrate` procedure. ({issue}`17006`)
I'd love to see this PR merged as well (: I'm currently unable to migrate our most important table due to this bug.
@marcinsbd please update the PR description with the information how timestamp information is handled in Hive tables converted to iceberg. especially whether "parquet timezone" setting gets applied to Iceberg.
cc @dain @electrum @jirassimok for "parquet timezone" context.
This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua
This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua
Closing this pull request, as it has been stale for six weeks. Feel free to re-open at any time.
I am reopening and adding stale-ignore since this PR seems to just have fallen through the cracks. Ideally it can be rebased and merge. Fyi @ebyhr @cwsteinbach @alexjo2144
https://github.com/trinodb/trino/pull/22781 looks very related and is now merged. @marcinsbd please rebase so that @raunaqmorarka can review here
#22781 looks very related and is now merged. @marcinsbd please rebase so that @raunaqmorarka can review here
done
Can we move this forward @ebyhr @cwsteinbach @alexjo2144 @raunaqmorarka @findepi ?