apd icon indicating copy to clipboard operation
apd copied to clipboard

Precision clarification

Open caycaycarly opened this issue 3 years ago • 1 comments

I've been using the lib with a precision of 34, and I've noticed that when using Text(), some of my decimals terminate with ...9999999 or ..0000001. I suspect it's got something to do with this.

Is there a way I can avoid having such errors, because my PoC app requires a certain sum total to be exactly zero, even at the precision of 34, which I can't achieve with such errors. I have to use f and can't use scientific notation. I'd like to know why this error occurs, and why it can't be fixed, because I've noticed the shopspring lib having a similar problem.

caycaycarly avatar Jul 24 '22 06:07 caycaycarly

Precision sets the number of decimals apart from contiguous zero decimals right after the decimal point. How do I truncate the number of decimals? That would solve my problem https://go.dev/play/p/oYrTqqzgJpB

caycaycarly avatar Jul 24 '22 09:07 caycaycarly