json-schema-faker-cli icon indicating copy to clipboard operation
json-schema-faker-cli copied to clipboard

Handling of large contract file with JSONStream

Open akrysiak-equinix opened this issue 1 year ago • 1 comments

Currently to read (and probably generate json) the tool uses: JSON.parse. For large contract files parsing ends up with exception:

SyntaxError: Expected ':' after property name in JSON at position 65536 (line 2154 column 19)
    at JSON.parse (<anonymous>)
    at generate (/usr/local/lib/node_modules/json-schema-faker-cli/node_modules/json-schema-faker/bin/gen.cjs:93:55)
    at Transform.transform [as _transform] (/usr/local/lib/node_modules/json-schema-faker-cli/node_modules/json-schema-faker/bin/gen.cjs:111:7)

The JSON parses stops after reaching max position: 65536

Would be feasible to use something like: https://www.npmjs.com/package/JSONStream?

akrysiak-equinix avatar Nov 14 '24 16:11 akrysiak-equinix

Thanks for reporting this.

Maybe streaming could make sense in handing large files. However, I'm not interested too much in maintaining this tool. I believe I haven't used it for many years. Especially large files aren't something valuable for me here because I've always used this tool for relatively short files, and I don't know about anybody else using large schemas. Instead of an enormous file, you could try splitting it into many shorter files.

Anyway, you're welcome to create your own fork.

oprogramador avatar Nov 15 '24 20:11 oprogramador