jsonlines icon indicating copy to clipboard operation
jsonlines copied to clipboard

Suggestion: Invalid lines are ignored, or more easily implemented, lines must start with trimmed '{' or '['

Open jvlake opened this issue 1 year ago • 2 comments

Suggestion: Invalid lines are ignored, or more easily implemented -> lines must start with trimmed '{' or '['

I love this format but the restriction that every line MUST be valid json would be better if invalid lines were just ignored.

Implementing an invalid-json-checker that actually checks the validity of each line has too great of a complexity for the specification, but adding a very simple rule. lines that don't start with '{' or '[' are ignored from processing. now that's cooking with gas.

If this breaks the strict "jsonlines standard" maybe call it jsonlines-loose.

---- begin of example jsonlines.jsonll ---- 
# I can now be a comment because the row will be skipped.
{ "msg": "I'll be processed" }
{ "msg": "I'll blow up by whatever parses the row as json is not closed ->"
    {"msg":"I can be a legit  row because string.trim() is simple in most languages and starts with a known good character"}
---- end of example jsonlines.jsonll ---- 

jvlake avatar Mar 01 '23 09:03 jvlake

Adding comments/ignored lines makes it more difficult to tell which line of a file corresponds to which record. Without comments, record number == line number, making using unix text tools much easier.

The suggestion here would also silently ignore some records that used to be processed (ones starting with whitespace, plain strings, numbers, boolean, null values) and makes implementation more complicated.

wardi avatar Mar 01 '23 15:03 wardi

Thanks for replying. The point about unix editors and line numbers is legit.

After reading https://www.json.org/json-en.html I see what you mean; that the json that doesn't use 'object syntax' is still valid. Seems a bit crazy as it's jsObjectNotation not jsTheValueOfAnything, and the spec says "An object is an unordered set of name/value pairs"

Any who, it was only a suggestion and certainly a decent change from your spec. Maybe I'll fork the spec and make my own loose version because it's just too useful.

jvlake avatar Mar 02 '23 09:03 jvlake