[Java] Abstract class corner cases failing: arrays and enums

[gudzpoz]

Search before asking

• [x] I had searched in the issues and found no similar issues.

Version

OS: ArchLinux (6.16.7-zen1-1-zen) Fory: 'org.apache.fory:fory-core:0.13.0-SNAPSHOT' (Sep 30) Failing JDK: GraalVM 23.0.2+7.1 (build 23.0.2+7-jvmci-b01)

Component(s)

Java

Minimal reproduce step

Corner case 1: AbstractObject[]: https://github.com/gudzpoz/fory-native-image-reproducer/blob/abstract-array/app/src/main/java/org/example/App.java

$ git clone https://github.com/gudzpoz/fory-native-image-reproducer.git
$ git checkout abstract-array
$ ./gradlew run
Caused by: org.apache.fory.exception.InsecureException: class org.example.App$AbstractObject is not registered, please check whether it's the type you want to serialize or a **vulnerability**. If safe, you should invoke `Fory#register` to register class,  which will have better performance by skipping classname serialization. If your env is 100% secure, you can also avoid this exception by disabling class registration check using `ForyBuilder#requireClassRegistration(false)`

Corner case 2: Abstract enums: (failing only under native images) https://github.com/gudzpoz/fory-native-image-reproducer/blob/abstract-enums/app/src/main/java/org/example/App.java

$ git clone https://github.com/gudzpoz/fory-native-image-reproducer.git
$ git checkout abstract-enum
$ ./gradlew run # pass
$ ./gradlew nativeCompile
$ app/build/native/nativeCompile/test
java.lang.RuntimeException: Class class org.example.App$MyEnum is not registered

What did you expect to see?

The tests should pass.

What did you see instead?

Corner case 1: AbstractObject[]:

Exception in thread "main" java.lang.ExceptionInInitializerError
Caused by: org.apache.fory.exception.InsecureException: class org.example.App$AbstractObject is not registered, please check whether it's the type you want to serialize or a **vulnerability**. If safe, you should invoke `Fory#register` to register class,  which will have better performance by skipping classname serialization. If your env is 100% secure, you can also avoid this exception by disabling class registration check using `ForyBuilder#requireClassRegistration(false)`
        at org.apache.fory.resolver.ClassResolver.createSerializer(ClassResolver.java:1220)
        at org.apache.fory.resolver.ClassResolver.getClassInfo(ClassResolver.java:1135)
        at org.apache.fory.resolver.ClassResolver.createSerializer0(ClassResolver.java:1269)
        at org.apache.fory.resolver.ClassResolver.lambda$ensureSerializersCompiled$14(ClassResolver.java:1803)
        at org.apache.fory.collection.ForyObjectMap.forEach(ForyObjectMap.java:302)
        at org.apache.fory.resolver.ClassResolver.ensureSerializersCompiled(ClassResolver.java:1796)
        at org.apache.fory.Fory.ensureSerializersCompiled(Fory.java:1606)
        at org.example.App.<clinit>(App.java:20)

Corner case 2: Abstract enums: (failing only under native images)

Exception in thread "main" org.apache.fory.exception.SerializationException: java.lang.RuntimeException: Class class org.example.App$MyEnum is not registered
	at org.apache.fory.Fory.processSerializationError(Fory.java:360)
	at org.apache.fory.Fory.serialize(Fory.java:329)
	at org.apache.fory.Fory.serialize(Fory.java:277)
	at org.example.App.main(App.java:31)
	at java.base@25/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)
Caused by: java.lang.RuntimeException: Class class org.example.App$MyEnum is not registered
	at org.apache.fory.resolver.TypeResolver.getSerializerClassFromGraalvmRegistry(TypeResolver.java:635)
	at org.apache.fory.resolver.ClassResolver.getSerializerClass(ClassResolver.java:862)
	at org.apache.fory.resolver.ClassResolver.getSerializerClass(ClassResolver.java:854)
	at org.apache.fory.resolver.ClassResolver.createSerializer(ClassResolver.java:1260)
	at org.apache.fory.resolver.ClassResolver.getOrUpdateClassInfo(ClassResolver.java:1182)
	at org.apache.fory.resolver.ClassResolver.getRawSerializer(ClassResolver.java:847)
	at org.example.AppForyRefCodec_0.<init>(AppForyRefCodec_0.java:38)
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.methodhandles.Util_java_lang_invoke_MethodHandle.invokeInternal(Target_java_lang_invoke_MethodHandle.java:241)
	at java.base@25/java.lang.invoke.MethodHandle.invokeBasic(MethodHandle.java:102)
	at java.base@25/java.lang.invoke.MethodHandle.invokeBasic(MethodHandle.java:0)
	at java.base@25/java.lang.invoke.Invokers$Holder.invoke_MT(Invokers$Holder)
	at org.apache.fory.serializer.Serializers.newSerializer(Serializers.java:101)
	at org.apache.fory.resolver.ClassResolver.createSerializer(ClassResolver.java:1261)
	at org.apache.fory.resolver.ClassResolver.getOrUpdateClassInfo(ClassResolver.java:1182)
	at org.apache.fory.Fory.write(Fory.java:401)
	at org.apache.fory.Fory.serialize(Fory.java:323)
	... 3 more

Anything Else?

This seems to fix the issues, but I'm not entirely sure how enums are serialized:

diff --git c/java/fory-core/src/main/java/org/apache/fory/resolver/ClassResolver.java w/java/fory-core/src/main/java/org/apache/fory/resolver/ClassResolver.java
index 8b9eca8d..b36cc1ea 100644
--- c/java/fory-core/src/main/java/org/apache/fory/resolver/ClassResolver.java
+++ w/java/fory-core/src/main/java/org/apache/fory/resolver/ClassResolver.java
@@ -1800,7 +1800,10 @@ public class ClassResolver extends TypeResolver {
                 createSerializer0(cls);
               }
               if (cls.isArray()) {
-                createSerializer0(TypeUtils.getArrayComponent(cls));
+                Class<?> component = TypeUtils.getArrayComponent(cls);
+                if (isSerializable(component)) {
+                  createSerializer0(component);
+                }
               }
             }
           });
diff --git c/java/fory-core/src/main/java/org/apache/fory/resolver/TypeResolver.java w/java/fory-core/src/main/java/org/apache/fory/resolver/TypeResolver.java
index 4db04ecf..ea62ab3b 100644
--- c/java/fory-core/src/main/java/org/apache/fory/resolver/TypeResolver.java
+++ w/java/fory-core/src/main/java/org/apache/fory/resolver/TypeResolver.java
@@ -444,6 +444,9 @@ public abstract class TypeResolver {
   public abstract ClassDef getTypeDef(Class<?> cls, boolean resolveParent);
 
   public final boolean isSerializable(Class<?> cls) {
+    if (cls.isEnum()) {
+      return true;
+    }
     if (ReflectionUtils.isAbstract(cls) || cls.isInterface()) {
       return false;
     }

Are you willing to submit a PR?

• [x] I'm willing to submit a PR!

[rahuldas-404]

Hi @gudzpoz , I'd love to try working on this issue. Could you please assign it to me?

[rahuldas-404]

@gudzpoz @chaokunyang if no one is working, please assign this to me

[gudzpoz]

I don't think I can assign issues. @chaokunyang Would you pleaese assign them this?

[chaokunyang]

@rahuldas-404 assigned to you, thanks for contributing to apache fory