gc-excelviewer icon indicating copy to clipboard operation
gc-excelviewer copied to clipboard

Large numbers shows incorrectly

Open Mingun opened this issue 2 years ago • 0 comments

For example, in this CSV all numbers displayed as in the last column:

"No","Actual number","JS truncation","Shows as"
1,"210727301071531090","210727301071531100","210727301071531104"
2,"210727301151031138","210727301151031140","210727301151031136"
3,"210825582544337195","210825582544337180","210825582544337184"

This seems to the JS number truncation due to that fact that JS numbers have double floating-point representation with only 52 bit of mantissa so not every 64-bit number can be exactly represented as JS number. The "JS truncation" columns shows the result from Firefox console for numbers. But this plugin also adds an own systematic error of ±4 as you can see in the last column.

Extension info

Имя: Excel Viewer Идентификатор: grapecity.gc-excelviewer Описание: View Excel spreadsheets and CSV files in Visual Studio Code and VS Code for the Web. Версия: 4.0.50 Издатель: GrapeCity Ссылка на Visual Studio Marketplace: https://marketplace.visualstudio.com/items?itemName=GrapeCity.gc-excelviewer

Mingun avatar Jan 25 '22 09:01 Mingun