vscode-q icon indicating copy to clipboard operation
vscode-q copied to clipboard

Extension incorrectly parses int64 type

Open Yanpas opened this issue 1 year ago • 6 comments

We store in one column int64 values.

In grid view last two digits become zeroes. In visualization they are rendered as int32_min.

E.g. 1608532083302613811 in grid view becomes 1608532083302613800. In visualization: -2,147,483,648

Yanpas avatar Feb 06 '24 16:02 Yanpas

It is because javascript doesn't support int64 type. You have to convert it to string before load them in grid or virtualization mode.

jshinonome avatar Feb 07 '24 01:02 jshinonome

The bad thing is that extension silently misleads about the data that's contained in the table (this big number in the example was an id).

JS has BigInt class to deal with larger integer types.

Also I'm fine with treating it as string but I guess sorting will work incorrect then.

Yanpas avatar Feb 07 '24 01:02 Yanpas

The underlying lib node-q doesn't use bigint. Though my lib (jkdb) uses bigint, sending json message to grid or virtualization still has to convert bigint to general number. I will think if it is possible to send arrow format. However, I am less likely to implement the feature for vscode-q. You could try kx's official plugin and ask them to fix it.

jshinonome avatar Feb 07 '24 01:02 jshinonome

This is already available in vscode-q from a quick overlook: https://github.com/michaelwittig/node-q?tab=readme-ov-file#types-footnote5

There's a long2number parameter of q.ConnectionParameters

Yanpas avatar Feb 07 '24 01:02 Yanpas

Grid/virtualization mode is an independent html page from vscode-q conn manager, when vscode-q got data from kdb, it has to send data to grid/virtualization html page. The data conversion caused losing precision. Sending as string data type to grid/virtualization page, the chart cannot display properly.

jshinonome avatar Feb 07 '24 01:02 jshinonome

Side question: do debug breakpoints for typescript code of the extension (client) work in your environment? Looks like mine vscode can't find sourcemaps: image The diagnostic tells:

We couldn't find a corresponding source location, and didn't find any source with the name client.ts.
How did you expect this file to be loaded? (If you have a compilation step, you should pick 'sourcemap')

I've tried both Webpack & Launch Extension and npx tsc and then Launch Extension

PS want to experiment with https://github.com/sidorares/json-bigint or encoding everything as an object with manual deserialization on webview side

Yanpas avatar Feb 08 '24 01:02 Yanpas