nray icon indicating copy to clipboard operation
nray copied to clipboard

nray-output.json is malformatted JSON

Open MexHigh opened this issue 1 year ago • 3 comments

Example nray-output.json

{"nodeID":"e606aa76","nodeName":"e606aa76","timestamp":"2023-04-12T09:45:46.624838Z","scannername":"native-portscanner","result":{"target":"172.16.23.10","port":139,"portscan":{"target":"172.16.23.10","port":139,"open":true,"scantype":"tcpconnect","timeout":1000}}}
{"nodeID":"e606aa76","nodeName":"e606aa76","timestamp":"2023-04-12T09:45:46.624838Z","scannername":"native-portscanner","result":{"target":"172.16.23.2","port":139,"portscan":{"target":"172.16.23.2","port":139,"open":true,"scantype":"tcpconnect","timeout":1000}}}
{"nodeID":"e606aa76","nodeName":"e606aa76","timestamp":"2023-04-12T09:45:48.214239900Z","scannername":"native-portscanner","result":{"target":"172.16.23.2","port":49208,"portscan":{"target":"172.16.23.2","port":49208,"open":true,"scantype":"udp","timeout":1000}}}

Each line of the file has it's own JSON document, which for some reason makes sense, but this is a characteristic of a log file, not a JSON file. It is therefore unparsable for external software.

Is it possible that you add enclosing JSON array brackets to the file so that it can be parsed as JSON?

MexHigh avatar Apr 12 '23 09:04 MexHigh

You're right, it is rather JSONL than JSON, see https://jsonlines.org/ I made this choice because I'm ultra annoyed by malformed partial JSON (or XML or any other format) when interrupting tools. All progress is lost unless you do a deep dive in adding correct closing brackets and tags and so on. In contrast, even when Ctrl+C-ing nray, the stuff that has already been written is fully usable.

I also use jq primarily for handling the data, which works fine with JSONL. I therefore like to stick with this format, as manually enclosing it with array brackets is less of a hassle then recovering a broken JSON structure.

edermi avatar Apr 12 '23 12:04 edermi

Well this makes sense. Thanks for the clarification. Is there any way to "announce" this in the (default) filename itself, e.g. by its extension? Is there a well-known .jsonl extenstion?

MexHigh avatar Apr 13 '23 18:04 MexHigh

Honestly I never did a deep dive on the topic.

  • Wikipedia has something in its article about JSON streaming (TIL the term NDJSON): https://en.wikipedia.org/wiki/JSON_streaming#Newline-Delimited_JSON
  • Apparently there has been a RFC draft which already expired: https://www.ietf.org/archive/id/draft-hallambaker-jsonl-01.txt

Technically nothing prevents you from giving your output another file extension, it's a setting in the config file. The documentation may be more clear on this topic, I'll take this feedback with me and try to come up with improvements the next time working on the code base.

I'll leave the ticket open until then, thanks for your feedback!

edermi avatar Apr 14 '23 08:04 edermi