delta-rs
delta-rs copied to clipboard
Support parquet files with nanoseconds timestamps when converting a Parquet table to a Delta table
Description
When converting from ArrowSchema
to deltalake::Schema
, the try_from
method will return an error if the arrow schema has a Timestamp(nanosecond)
. This causes the convert_to_delta
function to fail. It makes sense to support parquet files with nanoseconds timestamps when converting a Parquet table to a Delta table.
Use Case
Convert parquet files in rust/tests/data/delta-1.2.1-only-struct-stats
to a Delta table.
Related Issue(s) https://github.com/delta-io/delta-rs/issues/1041, https://github.com/delta-io/delta-rs/issues/1682, https://github.com/delta-io/delta-rs/pull/1686