Parsing null in array fails for JValue

[vdichev]

parse[JValue]("""[null]""") produces "Can not deserialize instance of com.codahale.jerkson.AST$JValue out of VALUE_NULL token" at ... blablabla

[sroebuck]

Coda answered this one for me today…

Just use: parseOption[JValue]

The result will then be null.

Simple when you know how! :)

Okay, I was in too much of a rush last night… read on for clarification

[vdichev]

Sorry, but this doesn't work on 0.5.0 and it doesn't make much sense- is this supposed to also handle nulls in any nested arrays?

Even if it worked, why would you get null for anything which is declared to be an Option? Shouldn't it be None?

[vdichev]

If you mean this answer: https://twitter.com/coda/status/141959832570834944 there are use cases where it's not applicable.

To give more context: of course it would work with parse[List[Option[JValue]]]("""[null]"""). The problem is that you have to know you'll get an array. Sometimes you don't: it could be a map or an array. And this is valid for nested structures as well. Even if you know you get an array, its elements in turn could be arrays, and any nulls inside them will cause an exception.

The way I see it, JValue could be anything, and if parse[JValue]("null") and parse[JValue]("""[1]""") work, then so should parse[JValue]("""[null]"""). It should only crash if I've specified something which the parsed string is not, like parse[List[Int]]("""{"hey":3}""")

[sroebuck]

Apologies, I was in too much of a rush last night that I didn't test my example and also typed null instead of None which was what I was expecting the result to be.

Please note that these comments below represent my understanding of what happens and (at least for me) a workable solution but clearly, in time, it would be good if Jerkson made these things work better.

So, let's get it right this time. First of all the preamble:

scala> import com.codahale.jerkson.Json
scala> import com.codahale.jerkson.AST.JValue

Okay, as you say

scala> Json.parse[JValue]("""[null]""")

produces:

com.codahale.jerkson.ParsingException: Can not deserialize instance of com.codahale.jerkson.AST$JValue out of VALUE_NULL token
 at [Source: java.io.StringReader@63041bb7; line: 1, column: 2]

I thought this should work as I thought JValue could also be able to represent null, indeed I still think it or some superclass should, and if you enter:

scala> Json.parse[JValue]("""null""")  // <- Note the removal of the square brackets

The result is:

com.codahale.jerkson.AST.JValue = JNull

So a null on its own does have a JValue representation and parses correctly.

Now, Coda's tweet: https://twitter.com/coda/status/141959832570834944 prompted me to try:

scala> Json.parse[Option[JValue]]("""[null]""")

But unfortunately this doesn't work either, giving effectively the same error:

com.codahale.jerkson.ParsingException: Can not deserialize instance of com.codahale.jerkson.AST$JValue out of VALUE_NULL token
 at [Source: java.io.StringReader@3b18618d; line: 1, column: 2]

However, if you are in a position to anticipate the top level collection of the object you are parsing. And in working cases this is true, then you can enter:

scala> Json.parse[List[Option[JValue]]]("""[null]""")

So, here we are saying that we expect a list of anything which may also contain nulls ( - e.g. they are optional). And the result is:

List[Option[com.codahale.jerkson.AST.JValue]] = List(None)

which feels like progress. And it does, to some extent allow for completely unknown JSON hierarchies. For example:

scala> Json.parse[List[Option[JValue]]]("""[null,{"a":"map", "with":["a","list"]}]""")

correctly returns:

List[Option[com.codahale.jerkson.AST.JValue]] = List(None, Some(JObject(List(JField(a,JString(map)), JField(with,JArray(List(JString(a), JString(list))))))))

However, you always have to be on the lookout for nulls which, if not anticipated by an explicit Option in the type definition cause our favourite error:

scala> Json.parse[List[Option[JValue]]]("""[null,{"a":"map", "with":["a","list",null]}]""")
    // Note null added to end of this embedded list                             ^^^^

Which results in:

com.codahale.jerkson.ParsingException: Can not deserialize instance of com.codahale.jerkson.AST$JValue out of VALUE_NULL token
 at [Source: java.io.StringReader@2973cb2c; line: 1, column: 37]

I hope that explains and I apologise for yesterdays rush response.

[vdichev]

Thanks, Stuart, this explains it better than I did.

As you mentioned, this workaround is not a general solution, since "you always have to be on the lookout for nulls".