datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

Optimization InList: compare the float data type using OrderedFloat<T>

Open liukun4515 opened this issue 3 years ago • 0 comments

Elsewhere in DataFusion (including in ScalarValue) we use ordered_float to compare floating point numbers

It might be possible to use set::<OrderedFloat<f32>>, which would be more space efficient (fewer bytes than ScalarValue) as well as faster (as the comparison doens't have to dispatch on the type each time)

https://github.com/apache/arrow-datafusion/blob/88b88d4360054a85982987aa07b3f3afd2db7d70/datafusion/common/src/scalar.rs#L33

Originally posted by @alamb in https://github.com/apache/arrow-datafusion/pull/2809#discussion_r912341240

liukun4515 avatar Jul 03 '22 03:07 liukun4515