delta-rs
delta-rs copied to clipboard
Support `Time32` and `Time64` types
Description
The delta-rs PrimitiveType supports timestamp and date, but not time. This means that Arrow Time32 and Time64 types are not supported.
Use Case
I would like to use Time32 and Time64 types in my DataFusion DeltaTables.
Related Issue(s)
take
Are other types possible to add as well, such as Duration? I had a source the other day where most columns were duration format in Polars (HH:MM:SS) but it failed to save to the Delta Table. I converted to an integer for now.
pl.col("response_sla").dt.total_milliseconds().alias("response_sla_ms"),
All these types are not supported by the delta protocol, so we can't add them.
For duration types i actually made an upstream request at the delta io repo, so you could up vote or ping the databricks folks there to make it happen