vscode-data-preview icon indicating copy to clipboard operation
vscode-data-preview copied to clipboard

Incorrect type inferencing for parquet files

Open Conor-Yates-Koch opened this issue 3 years ago • 1 comments

I have a parquet file with four columns, all of which are string type. However, some of the values for one of the columns (particularly in the first rows) contain strings with only numeric characters.

The extension infers the type of this column to be numeric, causing two issues:

  • values are incorrectly displayed, containing thousands separators and two superfluous decimal places
  • values later in the file which do contain alphabetic characters are not displayed at all, instead being displayed in the cell with the - character.

Conor-Yates-Koch avatar Oct 11 '22 19:10 Conor-Yates-Koch

that might be the case as data preview uses older version of Perspective viewer and old parquet data loading library I could find that worked at the time.

See this update for more info: https://github.com/RandomFractals/vscode-data-preview/issues/316#issuecomment-1277766785

RandomFractals avatar Oct 13 '22 15:10 RandomFractals