datafusion
datafusion copied to clipboard
Optimization InList: compare the float data type using OrderedFloat<T>
Elsewhere in DataFusion (including in ScalarValue) we use ordered_float to compare floating point numbers
It might be possible to use set::<OrderedFloat<f32>>, which would be more space efficient (fewer bytes than ScalarValue) as well as faster (as the comparison doens't have to dispatch on the type each time)
https://github.com/apache/arrow-datafusion/blob/88b88d4360054a85982987aa07b3f3afd2db7d70/datafusion/common/src/scalar.rs#L33
Originally posted by @alamb in https://github.com/apache/arrow-datafusion/pull/2809#discussion_r912341240