enumeratum
enumeratum copied to clipboard
lloydmeta
NullPointerException during JSON deserialization
I'm seeing this very hard to reproduce NullPointerException when I'm trying to deserialize JSON with play-json into an enumeratum enum.
java.lang.NullPointerException: null
at enumeratum.Enum$$anonfun$lowerCaseNamesToValuesMap$1.apply(Enum.scala:47)
at enumeratum.Enum$$anonfun$lowerCaseNamesToValuesMap$1.apply(Enum.scala:47)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at enumeratum.Enum$class.lowerCaseNamesToValuesMap(Enum.scala:47)
It appears the the value of lowerCaseNamesToValuesMap is null. My guess is that it is connected to the variable being declared lazy.
If I access the variable lowerCaseNamesToValuesMap before the deserilisation the problem goes away. I used to only see this in test code but now also in production code.
Have you seen anything like this before?
Strange, I've not seen this in my usage of Enumeratum per se, but I have seen something similar when I've had cyclic dependencies between different objects. I don't remember the specifics, but I do remember it was hard to track down because it was non-obvious, but I think I resolved the problem by using lazy val though. I think it's unlikely that there's a NPE bug inherent to the usage of lazy val because it's such a core part of the language..
Can you put up a minimal example that reproduces the problem so we can take a look together?
I'm going to try to extract a minimal example from my large-ish application. Not sure when I will get around to doing this.
Hello, I've faced with the same error, so far hard to determine what part introduces the error. BTW I've managed to extract a minimal example to reproduce, might be some are excessive...
Could you look at the test in the attachment? error-example.tar.gz
@sergey-tikhonenko can you throw that in a repo? Much easier to take a look then.
Please, look at https://github.com/sergey-tikhonenko/org.sandbox.enumeratum
Cool, thanks for that.
I think the culprit is the same as in #138 and #144; classic circular dependency going on between companion object and class init. I've submitted sergey-tikhonenko/org.sandbox.enumeratum#1 which demonstrates how this can be addressed.
I see the null exception with Jackson and deadlocks on the others when I run many tests in parallel that uses the case objects directly (with mixed Scala and Java code - not sure if that's related). If I assign the case objects to val declarations and use that in my tests, then the problem goes away. I'm using a 14-core CPU so it's pretty easy to reproduce. This is true especially when the enum has a large number of entries and it's probably related to the way Scala instantiates objects.
@mlvn23 interesting; would you happen to have any dependencies (references to each other) amongst the different enums? In any case, if you could share a minimised example, it would be very helpful :)
I have a project where there are ~3000 enum members for one of the enums (don't ask) and haven't gotten any NPEs in parallel testing or in prod. That said, this is a Play project using Play-JSON.
I'm encountering the same problem.
I also encountered a variation whereby a val I was declaring in my PlayEnum object also consistently had a null value in findValue (seemingly always the same one), I managed to work around this by declaring it lazy:
lazy val displayValues = findValues.foldLeft(SortedMap.empty[Category, List[Category]](DisplayOrdering)) { (z, b) =>
b.parentCategory match {
case Some(p) =>
z + (p -> (z.get(p).getOrElse(List.empty[Category]) :+ b))
case None =>
z
}
}.mapValues(v => v.sortBy(_.entryName))
As the OP mentioned, if I access Enum.namesToValuesMap and Enum.lowerCaseNamesToValuesMap beforehand (I do it on startup in an eager singleton). If I don't, the first time I submit a form and trying to parse the form into a domain object using Enum.withName(name) it throws the NullPointerException. Curiously this worked fine until I've reworked this particular enumeration a bit, adding some properties and additional enumeration constants (case object)
The problem I think is strictly coupled with supplemental params added in a enum class and the macro applyed with findValues. I solved the issue declaring lazy findValue method.
Encountered this. The problem was triggered under a large ScalaTest suite.
It seems the scenario includes two threads racing on the initialization of SomeEnum$, SomeEnum$.values, and the various instances of SomeEnum$.Foo$, SomeEnum$.Bar$, SomeEnum$.Etc$.
The problem is that when the various Foo/Bar/Etc values of SomeEnum are initialized by a different thread, the findValues macro might not see the value of Foo$.MODULE$ and accidentally use a null instead.
Happened under scala 2.12.8 on two distinct multicore Intel processors (i7-7700HQ which is 4C8T, and another of a similar vintage which is 2C4T).
Not sure whether the "lazy val" causes a proper read barrier to be in place to guarantee the visibility of the other thread's initialization of SomeEnum$.Foo$, or whether that invariant would hold in scala 2.13.0 (not in position to test/reproduce under 2.13 at this moment) or whether "lazy val" just causes enough delay that it now works by luck, but so far going with this.
Suggestion: the implementation of the findValues might want to strive to ensure the appropriate read barrier is in place before attempting to read each of the values' respective MODULE$ static variables. Possibly, this happens in https://github.com/lloydmeta/enumeratum/blob/master/macros/src/main/scala/enumeratum/EnumMacros.scala#L160 but might require a JVM-specific hack to locate the appropriate thing to synchronize on?
