ecma402 icon indicating copy to clipboard operation
ecma402 copied to clipboard

How should ToRawPrecision handle rounding?

Open pyrocat101 opened this issue 5 years ago • 1 comments

With the current ToRawPrecision spec, the operation ToRawPrecision(9.99, 1, 2) will return a record like this:

{ formattedString: '1.0', roundedNumber: 10, integerDigitsCount: 1 }

which is weird in that formattedString is 1.0 instead 10. Is that intentional?

pyrocat101 avatar May 27 '20 00:05 pyrocat101

The intended behavior would be,

{ formattedString: '10', roundedNumber: 10, integerDigitsCount: 2 }

I think the problem is in how e is calculated. In your example, e is forced to be 0, but it should actually be 1. Please refer to last year's version of the spec for how to correctly calculate e:

https://www.ecma-international.org/ecma-402/6.0/index.html#sec-torawprecision

This slipped through the editorial review. I'll mark this as a bug to fix.

sffc avatar May 27 '20 01:05 sffc