bigdecimal icon indicating copy to clipboard operation
bigdecimal copied to clipboard

Difference in behavior with and without BigDecimal#dev 'digit' argument

Open k77ch7 opened this issue 1 year ago • 0 comments

In code from test_div_gh220, the last digit of x/y is 8.

x = BigDecimal("1.0")
y = BigDecimal("3672577333.6608990499165058135986328125")
c = BigDecimal("0.272288343892592687909520102748926752911779209181321744700032723729015151607289998e-9")

x/y
-> 0.272288343892592687909520102748926752911779209181321744700032723729015151607289998e-9

On the other hand, if a digit argument is used (e.g., 81, 82, 83), the 81st digit becomes 7.

x.div(y, 81)
-> 0.272288343892592687909520102748926752911779209181321744700032723729015151607289997e-9

x.div(y, 82)
-> 0.2722883438925926879095201027489267529117792091813217447000327237290151516072899973e-9

x.div(y, 83)
-> 0.27228834389259268790952010274892675291177920918132174470003272372901515160728999726e-9

I am trying to align JRuby's BigDecimal behavior with MRI. I would appreciate if I know reason of described above.

k77ch7 avatar Oct 09 '23 06:10 k77ch7