datafusion
datafusion copied to clipboard
Allow for writing hive-style partitions with datatypes beyond just Utf8 and Dictionary
Is your feature request related to a problem or challenge?
Currently in demux::compute_partition_keys_by_row the only supported types for writing out partitions seems to be just DataType::Utf8 and DataType::Dictionary(_, _). I think there is opportunity to support a number of other DataTypes such as int/uint 8/32/64 types, Date32 (with a fixed format 'yyyy-MM-dd') and bool.
Describe the solution you'd like
Code and tests for writing out hive-style partitions includes additional datatypes beyond just utf8 and Dictionary
Describe alternatives you've considered
Cast field to utf8 prior to output.
Additional context
No response