datafusion
datafusion copied to clipboard
DataFusion serialization doesn't handle `ScalarValue::Dictionary, Binary, LargeBinary, Time64, IntervalMonthDayNano, Struct`
Describe the bug
While updating to the latest and greatest DataFusion pin in IOx (https://github.com/influxdata/influxdb_iox/pull/5672) I got the following error:
Error encoding expr as protobuf: Dictionary(Int32, Utf8(\"B\")) is invalid as a DataFusion scalar value"
I think the issue is that now more data type casting is done as part of logical planning (https://github.com/apache/arrow-datafusion/pull/3396) the types of some constants (ScalarValues ) have been coerced to the correct type resulting in serialized ScalarValue::Dictionary that were not there before
To Reproduce
Try to serialize/deserialize ScalarValue::Dictionary
Expected behavior it works without error!
Additional context https://github.com/influxdata/influxdb_iox/pull/5672
Types that need support:
- [ ]
ScalarValue::Dictionaryhttps://github.com/apache/arrow-datafusion/pull/3532 - [ ]
ScalarValue::Binary - [ ]
ScalarValue::LargeBinary - [ ]
ScalarValue::Time64 - [ ]
ScalarValue::IntervalMonthDayNano - [ ]
ScalarValue::Struct
I will fix this