delta-rs icon indicating copy to clipboard operation
delta-rs copied to clipboard

Support `Time32` and `Time64` types

Open rebasedming opened this issue 1 year ago • 3 comments

Description

The delta-rs PrimitiveType supports timestamp and date, but not time. This means that Arrow Time32 and Time64 types are not supported.

Use Case

I would like to use Time32 and Time64 types in my DataFusion DeltaTables.

Related Issue(s)

rebasedming avatar Mar 09 '24 16:03 rebasedming

take

rebasedming avatar Mar 09 '24 16:03 rebasedming

Are other types possible to add as well, such as Duration? I had a source the other day where most columns were duration format in Polars (HH:MM:SS) but it failed to save to the Delta Table. I converted to an integer for now.

            pl.col("response_sla").dt.total_milliseconds().alias("response_sla_ms"),

ldacey avatar Mar 10 '24 17:03 ldacey

All these types are not supported by the delta protocol, so we can't add them.

For duration types i actually made an upstream request at the delta io repo, so you could up vote or ping the databricks folks there to make it happen

ion-elgreco avatar Mar 10 '24 17:03 ion-elgreco