usql icon indicating copy to clipboard operation
usql copied to clipboard

Cannot extract json containing an array as start element

Open kurtng opened this issue 7 years ago • 2 comments

JsonExtractor cannot handle if json contains an array as top most object. Following does not work

input json: [{'groupId':'g1'},{'groupId':'g2'}] extractor with path: new Microsoft.Analytics.Samples.Formats.Json.JsonExtractor("$[*]")

kurtng avatar Feb 09 '18 20:02 kurtng

Thanks for reporting this. I think this may have been broken with an earlier check in to make the handling of large top-level documents more streaming and less DOM-based. I will take a look while time permits.

MikeRys avatar Feb 09 '18 23:02 MikeRys

@kurtng we are having the same issue.

Thing is that on the file JsonExtractor.cs (line 57), there is the following line:

if (reader.TokenType == JsonToken.StartObject)

For what I understand, the first reader.Read() will be a JsonToken.StartArray so it will skip the first read and the path won't work, instead the path will be applied in the second iteration when the SelectChildren(token, this.rowpath) is called.

We are trying to make some changes but still haven't found a nice generic solution.

I'll let you know if we have some news.

fe-rod avatar Apr 09 '18 20:04 fe-rod