deserialize fails with java.lang.NullPointerException

[andyczerwonka]

Search before asking

• [X] I had searched in the issues and found no similar issues.

Version

0.5.0-rc2

Component(s)

Java

Minimal reproduce step

We are attempting to replace the latest snapshot with rc2, and we're getting the following exception

org.apache.fury.exception.DeserializationException: Deserialize failed, read objects are: [<clipped stringified model>, null, scala.collection.generic.DefaultSerializationProxy@5d2828c9, scala.collection.IterableFactory$ToFactory@3a082ff4, null]
	at org.apache.fury.Fury.handleReadFailed(Fury.java:784)
	at org.apache.fury.Fury.deserialize(Fury.java:755)
	at org.apache.fury.Fury.deserialize(Fury.java:685)
	at org.apache.fury.ThreadLocalFury.deserialize(ThreadLocalFury.java:99)
	at io.citrine.mithril.modules.predictors.Predictor$.$anonfun$decode$1(Predictor.scala:111)
	at scala.util.Try$.apply(Try.scala:210)
	at io.citrine.mithril.modules.predictors.Predictor$.decode(Predictor.scala:107)
	at io.citrine.mithrilworkbench.Main$.$anonfun$tryToDecodeThisPredictor$1(Main.scala:334)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
	at scribe.LoggerSupport.elapsed(LoggerSupport.scala:67)
	at scribe.LoggerSupport.elapsed$(LoggerSupport.scala:62)
	at scribe.package$.elapsed(package.scala:8)
	at io.citrine.mithrilworkbench.Main$.tryToDecodeThisPredictor(Main.scala:323)
	at io.citrine.mithrilworkbench.Main$.$anonfun$main$5(Main.scala:351)
	at io.citrine.mithrilworkbench.Main$.$anonfun$main$5$adapted(Main.scala:351)
	at mainargs.Invoker$.$anonfun$invoke0$3(Invoker.scala:63)
	at mainargs.Result.flatMap(Result.scala:13)
	at mainargs.Result.flatMap$(Result.scala:12)
	at mainargs.Result$Success.flatMap(Result.scala:23)
	at mainargs.Invoker$.invoke0(Invoker.scala:62)
	at mainargs.Invoker$.invoke(Invoker.scala:72)
	at mainargs.Invoker$.$anonfun$runMains$1(Invoker.scala:92)
	at mainargs.Result.flatMap(Result.scala:13)
	at mainargs.Result.flatMap$(Result.scala:12)
	at mainargs.Result$Success.flatMap(Result.scala:23)
	at mainargs.Invoker$.groupArgs$1(Invoker.scala:92)
	at mainargs.Invoker$.runMains(Invoker.scala:107)
	at mainargs.ParserForMethods.runRaw0(Parser.scala:170)
	at mainargs.ParserForMethods.runEither(Parser.scala:108)
	at mainargs.ParserForMethods.runEither(Parser.scala:151)
	at mainargs.ParserForMethods.runOrExit(Parser.scala:56)
	at io.citrine.mithrilworkbench.Main$.main(Main.scala:352)
	at io.citrine.mithrilworkbench.Main.main(Main.scala)
Caused by: java.lang.NullPointerException
	at scala.collection.IterableFactory$ToFactory.newBuilder(Factory.scala:275)
	at scala.collection.generic.DefaultSerializationProxy.readObject(DefaultSerializationProxy.scala:48)
	at org.apache.fury.serializer.ObjectStreamSerializer.read(ObjectStreamSerializer.java:221)
	at org.apache.fury.serializer.ReplaceResolveSerializer.readObject(ReplaceResolveSerializer.java:316)
	at org.apache.fury.serializer.ReplaceResolveSerializer.read(ReplaceResolveSerializer.java:305)
	at org.apache.fury.Fury.readData(Fury.java:884)
	at org.apache.fury.serializer.ReplaceResolveSerializer.read(ReplaceResolveSerializer.java:284)
	at org.apache.fury.serializer.collection.CollectionSerializers$JDKCompatibleCollectionSerializer.read(CollectionSerializers.java:574)
	at org.apache.fury.Fury.readDataInternal(Fury.java:916)
	at org.apache.fury.Fury.readRef(Fury.java:831)
	at org.apache.fury.serializer.ObjectSerializer.readOtherFieldValue(ObjectSerializer.java:438)
	at org.apache.fury.serializer.ObjectSerializer.readAndSetFields(ObjectSerializer.java:382)
	at org.apache.fury.serializer.ObjectSerializer.read(ObjectSerializer.java:316)
	at org.apache.fury.Fury.readDataInternal(Fury.java:916)
	at org.apache.fury.Fury.readRef(Fury.java:831)
	at org.apache.fury.serializer.ObjectSerializer.readOtherFieldValue(ObjectSerializer.java:438)
	at org.apache.fury.serializer.ObjectSerializer.readAndSetFields(ObjectSerializer.java:382)
	at org.apache.fury.serializer.ObjectSerializer.read(ObjectSerializer.java:316)
	at org.apache.fury.Fury.readDataInternal(Fury.java:916)
	at org.apache.fury.Fury.readRef(Fury.java:818)
	at org.apache.fury.serializer.ObjectStreamSerializer$FuryObjectInputStream.readObjectOverride(ObjectStreamSerializer.java:693)
	at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:480)
	at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:447)
	at scala.collection.generic.DefaultSerializationProxy.readObject(DefaultSerializationProxy.scala:54)
	at org.apache.fury.serializer.ObjectStreamSerializer.read(ObjectStreamSerializer.java:221)
	at org.apache.fury.serializer.ReplaceResolveSerializer.readObject(ReplaceResolveSerializer.java:316)
	at org.apache.fury.serializer.ReplaceResolveSerializer.read(ReplaceResolveSerializer.java:305)
	at org.apache.fury.Fury.readData(Fury.java:884)
	at org.apache.fury.serializer.ReplaceResolveSerializer.read(ReplaceResolveSerializer.java:284)
	at org.apache.fury.serializer.collection.CollectionSerializers$JDKCompatibleCollectionSerializer.read(CollectionSerializers.java:574)
	at org.apache.fury.Fury.readDataInternal(Fury.java:916)
	at org.apache.fury.Fury.readRef(Fury.java:831)
	at org.apache.fury.serializer.ObjectSerializer.readOtherFieldValue(ObjectSerializer.java:438)
	at org.apache.fury.serializer.ObjectSerializer.readAndSetFields(ObjectSerializer.java:382)
	at org.apache.fury.serializer.ObjectSerializer.read(ObjectSerializer.java:316)
	at org.apache.fury.Fury.readDataInternal(Fury.java:916)
	at org.apache.fury.Fury.readRef(Fury.java:818)
	at org.apache.fury.Fury.deserialize(Fury.java:751)

What did you expect to see?

We expect it to work.

What did you see instead?

Caused by: java.lang.NullPointerException
	at scala.collection.IterableFactory$ToFactory.newBuilder(Factory.scala:275)
	at scala.collection.generic.DefaultSerializationProxy.readObject(DefaultSerializationProxy.scala:48)

Anything Else?

No response

Are you willing to submit a PR?

• [ ] I'm willing to submit a PR!

[chaokunyang]

Hi @andyczerwonka , could you provide reproduction code here? I'll debug into your code to find out what happened

[andyczerwonka]

Hi @andyczerwonka , could you provide reproduction code here? I'll debug into your code to find out what happened

Unfortunately we're in the same situation as we are here.

[LiangliangSui]

Can we improve the log system and insert some info and debug logs into the code? For this situation where reproducible code cannot be provided but is reproducible stably, users can dynamically configure the log level through ENV, and then upload the collected log, we can restore the problem through these logs.

But I'm also worried that too many insert points in the code will cause performance degradation.

WDYT? @chaokunyang

[chaokunyang]

Can we improve the log system and insert some info and debug logs into the code? For this situation where reproducible code cannot be provided but is reproducible stably, users can dynamically configure the log level through ENV, and then upload the collected log, we can restore the problem through these logs.

But I'm also worried that too many insert points in the code will cause performance degradation.

WDYT? @chaokunyang

The log will be in critical path. If the error happens occasionally, the users may won't enable such log level.

If the error happens frequently, we may not need the log to restore the error

[chaokunyang]

Hi @andyczerwonka , could you reproduce this error locally now?