JSOX icon indicating copy to clipboard operation
JSOX copied to clipboard

Current state of the art

Open haoadoreorange opened this issue 5 years ago • 15 comments

Hello,

Sorry if not the right place, I'm just not sure where else to ask the question. Is the project still active ? it seems broken on different things on my local environment.

haoadoreorange avatar Jul 09 '20 06:07 haoadoreorange

yes; I did recently break some things I think... A lot of work went into coverage and testing... I think it used to be more forgiving on unquoted strings... can you give me an example of something broken for you?

d3x0r avatar Jul 09 '20 15:07 d3x0r

Oh JSOX? I was thinking this was a different project; I don't know of any failure conditions - hence not a lot of updates? again though, if you have a simple test....

d3x0r avatar Jul 09 '20 15:07 d3x0r

I have big json file here with a large array inside. It just returns an empty string for the array...

haoadoreorange avatar Jul 09 '20 15:07 haoadoreorange

the version from NPM or git?

d3x0r avatar Jul 09 '20 15:07 d3x0r

I have a question regarding the benchmark. The JSOX parse result in 4.0648 means its 4x times slower than the default parse ?

haoadoreorange avatar Jul 09 '20 15:07 haoadoreorange

the version from NPM or git?

ah I installed it from NPM

haoadoreorange avatar Jul 09 '20 15:07 haoadoreorange

I haven't published a new version :) can you please verify git and I'll tag and pulbish?

d3x0r avatar Jul 09 '20 15:07 d3x0r

yes; JSOX ends up doing a lot more checks for some things; it has improved a bit... It does depend a lot on what sort of thing is being parsed though... so I started to kind of walk away from those. This shouldn't be that much slower; but I think because of the graceful string failure sometimes the paths are longer than they would be - like GatherString() is fast at going from a quote to a quote... it's a little more work to check each character...

d3x0r avatar Jul 09 '20 15:07 d3x0r

I see, I'm having problem with parsing BigInt. I'm dealing with big chunk of them so store all as string will potentially increase the traffice size. But then 3rd parser have performance trade-off. JSON with BigInt is real pain.

JSOX seems to perform best so far. However, I tested JSOX and noticed that on very big number it stills lose precision. jsox.parse('{"start": 1539786952342493550}') will return 1539786952342493400

Again this is the npm version so I don't know if anything has been fixed. I'm using my forked version of json-bigint. Made a PR already. It's a bit slower (5x times).

haoadoreorange avatar Jul 09 '20 15:07 haoadoreorange

bigint requires a 'n' suffix...

d3x0r avatar Jul 09 '20 15:07 d3x0r

I could maybe add a test for a larger value than N and force bignum conversion... (as long as it isn't a float/having a decimal or exponent)

d3x0r avatar Jul 09 '20 15:07 d3x0r

				case VALUE_NUMBER:
					if( ( val.string.length > 10 ) || ( val.string.length == 9 && val[0]>'2' ) && !exponent_digit && !exponent_sign && !decimal )
						isBigInt = true;

like line 360, when converting the 'value' to a value.... adding the if, and setting bigint would force precision for larger values?

d3x0r avatar Jul 09 '20 16:07 d3x0r

4_294_967_295 is 10 the '9' should be 10 also

d3x0r avatar Jul 09 '20 16:07 d3x0r

Ah I see, indeed in json-bigint it enventually tests the length

haoadoreorange avatar Jul 09 '20 16:07 haoadoreorange

bigint behavior changed slightly; published new version with old and new detections. Implemented user defined type support tests.

sack.vfs JSOX support evolved somewhat from this, and user types became incompatible.

Found failures reviving some references, and later reviving user types with references.

Don't foresee changes in the future.

d3x0r avatar Dec 21 '20 07:12 d3x0r