datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

Remove redundant`is_numeric` and other checks for DataType

Open alamb opened this issue 3 years ago • 5 comments

arrow-rs has some functions to check data type, and the logic is also in datafusion, for example, https://docs.rs/arrow/7.0.0/arrow/datatypes/enum.DataType.html#method.is_numeric

We should reduce the redundancy in DataFusion to make the codebase clear.

Originally posted by @liukun4515 in https://github.com/apache/arrow-datafusion/pull/1606#discussion_r787306626

alamb avatar Jan 19 '22 12:01 alamb

This is a good one -- just delete some code from DataFusion 👍

alamb avatar Jan 21 '22 18:01 alamb

is_numeric in arrow-rs seems not to be same as in arrow-datafusion because DataType::Decimal is not included.

HaoYang670 avatar Jan 27 '22 04:01 HaoYang670

is_numeric in arrow-rs seems not to be same as in arrow-datafusion because DataType::Decimal is not included.

@HaoYang670

In the PG or Spark, the decimal is included in the numeric. Because the decimal is in development, some codes may be not consistent.

liukun4515 avatar Jan 27 '22 06:01 liukun4515

Now we can focus on other issues, and deal with them after other stabilization.

liukun4515 avatar Jan 27 '22 06:01 liukun4515

At least, should rename this function, so that developers would not misuse this and DataType::is_numeric.

HaoYang670 avatar Sep 19 '22 23:09 HaoYang670