math
math copied to clipboard
RFC: add `getPrecision` method.
Hi,
Similar to getScale
, would you be on favor of adding a method getPrecision
?
I want to perform app side validation on BigDecimal before passing to Postgres numeric fields, and getPrecision
would make that easier.
Thanks!
Hi, what would be the difference between getScale()
and getPrecision()
?
hi @BenMorel
The PostgreSQL manual will say it better than me:
We use the following terms below: The precision of a numeric is the total count of significant digits in the whole number, that is, the number of digits to both sides of the decimal point. The scale of a numeric is the count of decimal digits in the fractional part, to the right of the decimal point. So the number 23.5141 has a precision of 6 and a scale of 4. Integers can be considered to have a scale of zero.
@BenMorel any interest? thanks!
@BenMorel
Working with precision is generally better than working with scale
Check this example:
(0.03/360)*8*30 = 0.02
Woking with scale of 10: 0.03/360 = 0.0000833333(3) 0.0000833333*8=0.0006666664(0) 0.0006666664*30=0.0199999920(0)
❌ 0.0199999920 == 0.02
Working with precision of 10: 0.03/360 = 0.00008333333333(3) 0.00008333333333*8=0.0006666666666(40) 0.0006666666666*30=0.01999999999(80)=0.02
✔0.02==0.02
@alexviar i'm not sure what your example shows but it's not precision of 10
Working with precision of 10: 0.03/360 = 0.00008333333333(3)
0.00008333333333(3)
has as precision of 16
@alexviar i'm not sure what your example shows but it's not precision of 10
Working with precision of 10: 0.03/360 = 0.00008333333333(3)
0.00008333333333(3)
has as precision of 16
It has a scale of 14 but a precision of 10. The precision refers to how many significant digits it has, not how many decimal places it has. https://www.logicbig.com/quick-info/programming/precision-and-scale.html
For example Big.js uses scale whereas Decimal.js uses precision
@alexviar i'm not sure what your example shows but it's not precision of 10
Working with precision of 10: 0.03/360 = 0.00008333333333(3)
0.00008333333333(3)
has as precision of 16
I did put in parentheses digits that was rounded, and underlined digits are the Periodo of decimal
ah right thanks, i forgot that leading zeros are not significant.
but also for storage purposes (postgres), 0 <= scale <= precision so, It has a scale of 14 but a precision of 10.
doesn't work, in practise.
anyway, the point of this of was to simple add a getPrecision()
method, similar to getScale
, nothing mnore
Hi @bendavies, sorry for the late reply. I'm not against adding a getPrecision()
method if it is useful to you, so please feel free to open a PR along with a few tests, in particular to clarify what happens for leading zeros.
@alexviar I'm not sure what the decimal period has to do with the precision? Anyway, you make me think that this would be a great addition to BigRational
: a way to output 10/3
as 3.3(3)
for example. Would you want to work on this?
I opened issue #74 to track the BigRational as decimal number with period separately.
@alexviar I'm not sure what the decimal period has to do with the precision?
I think I didn't explain myself well. I did put rounded decimals (and decimal period) in parentheses for illustrative purposes only. Precision is given by working with n significant digits instead of n decimal place:
Operation | Rounded to 10 significant digits | Rounded to 10 decimal places |
---|---|---|
0.03 | 0.03000000000 | 0.0300000000 |
360 | 360.0000000 | 360.0000000000 |
8 | 8.000000000 | 8.0000000000 |
30 | 30.00000000 | 8.0000000000 |
0.03/360 | 0.00008333333333 | 0.0000833333 |
(0.03/360)*8 | 0.0006666666666 | 0.0006666666 |
(0.03/360)*8*30 | 0.02000000000 | 0.0200000000 |
Now, if we take the rounded values to do subsequent operations, we have:
Operation | Rounded to 10 significant digits | Rounded to 10 decimal places |
---|---|---|
0.03 | 0.03000000000 | 0.0300000000 |
360 | 360.0000000 | 360.0000000000 |
8 | 8.000000000 | 8.0000000000 |
30 | 30.00000000 | 8.0000000000 |
0.03/360 | 0.00008333333333 | 0.0000833333 |
(0.03/360)*8 | 0.0006666666666 | 0.0006666664 |
(0.03/360)*8*30 | 0.02000000000 | 0.0199999920 |
Currently it can be circunvent by using BigRational
, but this workaround doesn't work with irrational opreations like square root for example (~~sqrt(2)*sqrt(2)
~~ Never mind, I had read an article with an example like this, but I can not found it)