JsonSurfer icon indicating copy to clipboard operation
JsonSurfer copied to clipboard

Supply an OutputStream to a SurfingConfiguration

Open arenger opened this issue 5 years ago • 2 comments

First: Good job with JsonSurfer! It's a helpful tool, for sure.

I am looking for a way to read from a large JSON file in a streaming fashion and write an excerpt from that file to a new file in a streaming fashion. The onValue method of the JsonPathListener receives an Object which was obtained in a streaming manner. This is great, but at that point the full Object is loaded in memory. Would it be possible to specify an OutputStream to a SurfingConfiguration, such that the matching portions of the incoming file could be written to the stream instead of memory?

Actually, I really need a different OutputStream for every excerpt that is matched by the specified JSON Path. Perhaps a new SurfingConfiguration.setStreamHandler method would receive a new OutputStreamHandler interface that two methods: OutputStream onNewOutput(ParsingContext context) and void end(OutputStream out)... or something. There are a few ways to do it, I'm sure.

Maybe there is already support for something like this. I looked but couldn't find any.

arenger avatar Apr 25 '19 23:04 arenger

This feature would require modification to jsurfer-core, probably need to implement a new kind of JsonCollector which to be inserted into the dispatcher. Tips here: https://github.com/jsurfer/JsonSurfer/blob/master/jsurfer-core/src/main/java/org/jsfr/json/SurfingContext.java#L79

If you are interested, feel free to contribute. I'm currently working another issue regarding JsonPath filter during my limited free time

wanglingsong avatar Apr 26 '19 15:04 wanglingsong

I certainly understand the limited free time. No problem. This issue isn't urgent, currently, but if it becomes urgent I might look into contributing to the project. Thanks.

arenger avatar Apr 26 '19 16:04 arenger