tedious icon indicating copy to clipboard operation
tedious copied to clipboard

writeParameterData with a large scale value won't write correct buffer value

Open MartinJLee opened this issue 4 years ago • 1 comments

writeParameterData with { precision:30, scale:15, value:43069.0122916667 }

will generate a wrong buffer value. see the unit test.

    it.only('correctly converts 'number' values (Precision > 28 and scale 15)', function() {
      const value = 43069.0122916667;
      const expected = Buffer.from('11013be3c8dbb58701000000000000000000', 'hex');
      const precision = 30;

      const type = TYPES.Decimal;
      const parameterValue = { value, precision, scale: 15 };

      const buffer = Buffer.concat([...type.generateParameterData(parameterValue, { useUTC: false })]);
      assert.deepEqual(buffer, expected);
    });

Basically, the following will generate a number larger than int64 max

      const value = Math.round(Math.abs(parameter.value * Math.pow(10, parameter.scale)));

I'm not sure if the following code should handle the number or not

      else {
        buffer.writeUInt8(17);
        buffer.writeUInt8(sign);
        buffer.writeUInt64LE(value);
        buffer.writeUInt32LE(0x00000000);
        buffer.writeUInt32LE(0x00000000);
      }

MartinJLee avatar Feb 26 '20 05:02 MartinJLee

Hi @MartinJLee, this is a known issue. Checkout #678.

The issue is that Many (but not all) of the data types supported in SQL Server do not map 1:1 to JavaScript types. This happens especially with large precision and scale, in which case the calculated value goes beyond what JavaScript can handle.

For e.g., the value you want to store is 43069012291666710000, but JavaScript / Node.js can only handle up to 9007199254740991. We will re-focus on #678 to see what changes we can make to mitigate this issue.

For now you could try to use the library big-number

IanChokS avatar Feb 26 '20 22:02 IanChokS