oap-mllib icon indicating copy to clipboard operation
oap-mllib copied to clipboard

[K8S] Setting spark.executor.extraClassPath=./oap-mllib-1.2.0.jar doesn't work

Open xwu99 opened this issue 3 years ago • 0 comments

Setting spark.executor.extraClassPath=./oap-mllib-1.2.0.jar doesn't work, will cause incompatible class. Need to change to absolut path spark.executor.extraClassPath=/upload/oap-mllib-1.2.0.jar

Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 4.0 failed 1 times, most recent failure: Lost task 0.0 in stage 4.0 (TID 12) (172.17.0.5 executor 1): java.io.InvalidClassException: org.apache.spark.ml.clustering.KMeans; local class incompatible: stream classdesc serialVersionUID = -2012617696161676502, local class serialVersionUID = 5831058393139799759 at java.base/java.io.ObjectStreamClass.initNonProxy(Unknown Source) at java.base/java.io.ObjectInputStream.readNonProxyDesc(Unknown Source) at java.base/java.io.ObjectInputStream.readClassDesc(Unknown Source) at java.base/java.io.ObjectInputStream.readClass(Unknown Source) at java.base/java.io.ObjectInputStream.readObject0(Unknown Source) at java.base/java.io.ObjectInputStream.defaultReadFields(Unknown Source) at java.base/java.io.ObjectInputStream.readSerialData(Unknown Source) at java.base/java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) at java.base/java.io.ObjectInputStream.readObject0(Unknown Source) at java.base/java.io.ObjectInputStream.readArray(Unknown Source) at java.base/java.io.ObjectInputStream.readObject0(Unknown Source)

xwu99 avatar Nov 13 '21 03:11 xwu99