datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

division decimal: lose precision when convert i128 to f64

Open liukun4515 opened this issue 3 years ago • 1 comments

But now I find a bug in the arithmetic operation. https://github.com/apache/arrow-datafusion/blob/c91efc27658e58264c4f346a5cfdec8810179e90/datafusion/physical-expr/src/expressions/binary.rs#L302 and https://github.com/apache/arrow-datafusion/blob/c91efc27658e58264c4f346a5cfdec8810179e90/datafusion/physical-expr/src/expressions/binary.rs#L307

In the divide method, we may lose of precision by converting the i128 to f64. We use the f64 to store the larger range of data, but we lose the precision.

I will try to figure out this issue and resolve this.

Originally posted by @liukun4515 in https://github.com/apache/arrow-datafusion/pull/2233#discussion_r851859744

liukun4515 avatar Apr 18 '22 04:04 liukun4515

related to #122

liukun4515 avatar Apr 18 '22 04:04 liukun4515

I wonder if this is now completed 🤔

alamb avatar Jul 17 '23 12:07 alamb

I think the code using f64 for decimal division is out-of-dated for a while. We are using i128 directly to process it. That being said I don't expect this is still an issue.

viirya avatar Jul 17 '23 17:07 viirya