subzero icon indicating copy to clipboard operation
subzero copied to clipboard

Serialization exceptions on application which uses hazelcast and subzero and is deployed on 2 Weblogic nodes

Open pmihalcin opened this issue 5 years ago • 0 comments

Given I register classes for serialization using SubZero

private void setupSerialization(final Config config) {
	config.getSerializationConfig()
			.setEnableCompression(true);
	SubZero.useForClasses(config, GetApplicationDataRequest.class, GetApplicationDataResponse.class);
}
public class GetApplicationDataRequest {

    protected String applicationCode;
    protected List<ApplicationDataSet> dataSet;
	
	// getters, setters, equals, hashcode, empty constructor, all args constructor
}

and when I deploy application to 2 weblogic nodes which form hazelcast cluster using tcp ip protocol,

I am sometimes getting exceptions

Caused by: com.hazelcast.nio.serialization.HazelcastSerializationException: com.esotericsoftware.kryo.KryoException: Unable to find class: et.mihalcin
.v9.oxm.ApplicationDetail^A^@^A^A\
Serialization trace:
dataSet (net.mihalcin.GetApplicationDataRequest)
        at com.hazelcast.internal.serialization.impl.SerializationUtil.handleException(SerializationUtil.java:63) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:193) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.record.ObjectRecordFactory.newRecord(ObjectRecordFactory.java:37) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.recordstore.AbstractRecordStore.createRecord(AbstractRecordStore.java:111) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.recordstore.AbstractEvictableRecordStore.createRecord(AbstractEvictableRecordStore.java:55) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.recordstore.DefaultRecordStore.putInternal(DefaultRecordStore.java:701) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.recordstore.DefaultRecordStore.set(DefaultRecordStore.java:680) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.operation.SetOperation.run(SetOperation.java:39) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.Operation.call(Operation.java:170) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:208) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:197) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:413) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:153) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:123) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.run(OperationThread.java:110) ~[hazelcast-3.11.1.jar:3.11.1]
        at ------ submitted from ------.(Unknown Source) ~[?:?]
        at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.resolve(InvocationFuture.java:127) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.resolveAndThrowIfException(InvocationFuture.java:79) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.AbstractInvocationFuture.get(AbstractInvocationFuture.java:162) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.proxy.MapProxySupport.invokeOperation(MapProxySupport.java:434) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.proxy.MapProxySupport.setInternal(MapProxySupport.java:501) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.proxy.NearCachedMapProxyImpl.setInternal(NearCachedMapProxyImpl.java:257) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.proxy.MapProxyImpl.set(MapProxyImpl.java:246) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.proxy.MapProxyImpl.set(MapProxyImpl.java:237) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spring.cache.HazelcastCache.put(HazelcastCache.java:114) ~[hazelcast-spring-3.11.1.jar:3.11.1]
Caused by: com.esotericsoftware.kryo.KryoException: Unable to find class: et.mihalcin.ApplicationDetail^A^@^A^A\
Serialization trace:
dataSet (net.mihalcin.GetApplicationDataRequest)
        at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:160) ~[kryo-4.0.0.jar:?]
        at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133) ~[kryo-4.0.0.jar:?]
        at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:693) ~[kryo-4.0.0.jar:?]
        at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:118) ~[kryo-4.0.0.jar:?]
        at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540) ~[kryo-4.0.0.jar:?]
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:709) ~[kryo-4.0.0.jar:?]
        at info.jerrinot.subzero.internal.strategy.TypedKryoStrategy.readObject(TypedKryoStrategy.java:31) ~[subzero-core-0.9.jar:?]
        at info.jerrinot.subzero.internal.strategy.KryoStrategy.read(KryoStrategy.java:85) ~[subzero-core-0.9.jar:?]
        at info.jerrinot.subzero.AbstractSerializer.read(AbstractSerializer.java:30) ~[subzero-core-0.9.jar:?]
        at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:48) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:187) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.record.ObjectRecordFactory.newRecord(ObjectRecordFactory.java:37) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.recordstore.AbstractRecordStore.createRecord(AbstractRecordStore.java:111) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.recordstore.AbstractEvictableRecordStore.createRecord(AbstractEvictableRecordStore.java:55) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.recordstore.DefaultRecordStore.putInternal(DefaultRecordStore.java:701) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.recordstore.DefaultRecordStore.set(DefaultRecordStore.java:680) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.operation.SetOperation.run(SetOperation.java:39) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.Operation.call(Operation.java:170) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:208) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:197) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:413) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:153) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:123) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.run(OperationThread.java:110) ~[hazelcast-3.11.1.jar:3.11.1]
Caused by: java.lang.ClassNotFoundException: et.mihalcin.ApplicationDetail^A^@^A^A\
        at weblogic.utils.classloaders.GenericClassLoader.findLocalClass(GenericClassLoader.java:1029) ~[com.bea.core.utils.classloaders.jar:12.2.1.3]
        at weblogic.utils.classloaders.GenericClassLoader.findClass(GenericClassLoader.java:990) ~[com.bea.core.utils.classloaders.jar:12.2.1.3]
        at weblogic.utils.classloaders.ChangeAwareClassLoader.findClass(ChangeAwareClassLoader.java:101) ~[com.bea.core.utils.classloaders.jar:12.2.1.3]
        at weblogic.utils.classloaders.GenericClassLoader.doFindClass(GenericClassLoader.java:611) ~[com.bea.core.utils.classloaders.jar:12.2.1.3]
        at weblogic.utils.classloaders.GenericClassLoader.loadClass(GenericClassLoader.java:543) ~[com.bea.core.utils.classloaders.jar:12.2.1.3]
        at weblogic.utils.classloaders.GenericClassLoader.loadClass(GenericClassLoader.java:496) ~[com.bea.core.utils.classloaders.jar:12.2.1.3]
        at weblogic.utils.classloaders.ChangeAwareClassLoader.loadClass(ChangeAwareClassLoader.java:71) ~[com.bea.core.utils.classloaders.jar:12.2.1.3]
        at weblogic.utils.classloaders.ChangeAwareClassLoader.loadClass(ChangeAwareClassLoader.java:53) ~[com.bea.core.utils.classloaders.jar:12.2.1.3]
        at java.lang.Class.forName0(Native Method) ~[?:1.8.0_181]
        at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_181]
        at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:154) ~[kryo-4.0.0.jar:?]
        at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133) ~[kryo-4.0.0.jar:?]
        at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:693) ~[kryo-4.0.0.jar:?]
        at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:118) ~[kryo-4.0.0.jar:?]
        at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540) ~[kryo-4.0.0.jar:?]
        at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:709) ~[kryo-4.0.0.jar:?]
        at info.jerrinot.subzero.internal.strategy.TypedKryoStrategy.readObject(TypedKryoStrategy.java:31) ~[subzero-core-0.9.jar:?]
        at info.jerrinot.subzero.internal.strategy.KryoStrategy.read(KryoStrategy.java:85) ~[subzero-core-0.9.jar:?]
        at info.jerrinot.subzero.AbstractSerializer.read(AbstractSerializer.java:30) ~[subzero-core-0.9.jar:?]
        at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:48) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:187) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.record.ObjectRecordFactory.newRecord(ObjectRecordFactory.java:37) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.recordstore.AbstractRecordStore.createRecord(AbstractRecordStore.java:111) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.recordstore.AbstractEvictableRecordStore.createRecord(AbstractEvictableRecordStore.java:55) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.recordstore.DefaultRecordStore.putInternal(DefaultRecordStore.java:701) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.recordstore.DefaultRecordStore.set(DefaultRecordStore.java:680) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.map.impl.operation.SetOperation.run(SetOperation.java:39) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.Operation.call(Operation.java:170) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:208) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:197) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:413) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:153) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:123) ~[hazelcast-3.11.1.jar:3.11.1]
        at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.run(OperationThread.java:110) ~[hazelcast-3.11.1.jar:3.11.1]

Very weird part of these stacktraces is java.lang.ClassNotFoundException: et.mihalcin.ApplicationDetail^A^@^A^A\ where first character of class name is missing and suffix is some random characters.

Please note, it doesn't occur consistenly. It sometimes manifests, sometimes not.

Do you know why this happens?

Can it be something wrong with this library?

pmihalcin avatar Jul 04 '19 18:07 pmihalcin