latest guava cause [ unexpected type reference on method: 16 ]

[GGGTTTTT]

trafficstars

guava: 33.2.1-jre, byte-buddy: 1.14.18

When using byte-buddy to define java agent, some classes in Guava, such as Joiner, can result in the following errors:

java.lang.IllegalStateException: Unexpected type reference on method: 16
	at net.bytebuddy.pool.TypePool$Default$TypeExtractor$MethodExtractor.visitTypeAnnotation(TypePool.java:8791)
	at net.bytebuddy.jar.asm.ClassReader.readMethod(ClassReader.java:1453)
	at net.bytebuddy.jar.asm.ClassReader.accept(ClassReader.java:745)
	at net.bytebuddy.jar.asm.ClassReader.accept(ClassReader.java:425)
	at net.bytebuddy.pool.TypePool$Default.parse(TypePool.java:880)
	at net.bytebuddy.pool.TypePool$Default.doDescribe(TypePool.java:864)
	at net.bytebuddy.pool.TypePool$Default$WithLazyResolution.access$001(TypePool.java:944)
	at net.bytebuddy.pool.TypePool$Default$WithLazyResolution.doResolve(TypePool.java:1042)
	at net.bytebuddy.pool.TypePool$Default$WithLazyResolution$LazyTypeDescription.delegate(TypePool.java:1111)
	at net.bytebuddy.description.type.TypeDescription$AbstractBase$OfSimpleType$WithDelegation.getModifiers(TypeDescription.java:8540)
	at net.bytebuddy.description.type.TypeDescription$AbstractBase$OfSimpleType$WithDelegation.getModifiers(TypeDescription.java:8540)
	at net.bytebuddy.matcher.ModifierMatcher.doMatch(ModifierMatcher.java:60)
	at net.bytebuddy.matcher.ModifierMatcher.doMatch(ModifierMatcher.java:27)
	at net.bytebuddy.matcher.ElementMatcher$Junction$ForNonNullValues.matches(ElementMatcher.java:249)
	at net.bytebuddy.matcher.ElementMatcher$Junction$Disjunction.matches(ElementMatcher.java:214)
	at net.bytebuddy.agent.builder.AgentBuilder$RawMatcher$ForElementMatchers.matches(AgentBuilder.java:1973)
	at net.bytebuddy.agent.builder.AgentBuilder$RawMatcher$Disjunction.matches(AgentBuilder.java:1861)
	at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:12504)
	at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:12464)
	at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1800(AgentBuilder.java:12173)
	at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$LegacyVmDispatcher.run(AgentBuilder.java:12873)
	at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$LegacyVmDispatcher.run(AgentBuilder.java:12811)
	at java.security.AccessController.doPrivileged(Native Method)
	at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doPrivileged(AgentBuilder.java)
	at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:12373)
	at sun.instrument.TransformerManager.transform(TransformerManager.java:188)
	at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:428)
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	at com.test.TestMain.main(TestMain.java:10)

However, this problem does not arise when using Guava versions not exceeding 33.2.0-jre, despite the fact that the Joiner has remained unchanged. test-agent.zip test-project.zip

[raphw]

This likely related to the javac version that was used. There has been a row of bugs related to type annotations, and Byte Buddy fails to parse those as they do not match the specification.

[GGGTTTTT]

Thank you very much for your response. If I do not need to enhance the Guava classes, would it be appropriate to ignore these error messages?

[raphw]

You can do this. If you exclude them by their package prefix, the class will not be resolved. You will need to include this matcher first.

[GGGTTTTT]

I found that I can't avoid this error by excluding package names related to Guava, as the error is triggered by ignoreMatcher. Can I directly ignore this error message?

[raphw]

Yes, you would need to define a custom ignore matcher that ignores types of this name before attempting to resolve the type description for any other complex property.

[AKHILKRISHNANKK]

@raphw Will there be any updates in the future versions to address this situation?

[raphw]

By updating, the error will disappear eventually. Incorrect class files will lead to errors. There is some robustness in Byte Buddy, but generally speaking, errors like that cannot be really picked up.

[AKHILKRISHNANKK]

@raphw We have updated the byte buddy to the latest version (1.15.0) and the issue still persists. Can we expect a fix for this issue in the upcoming byte buddy releases ?

[raphw]

This is an error in Guava. If you update the library, does this error still occur?

[dogourd]

@raphw I've done some testing on this issue, and it seems more likely to be related to javac.

On the Windows platform, Java versions 9, 10, and 11 result in bytecode that isn't compatible with Byte Buddy, while starting from Java 12, things return to normal. All versions tested were from Azul JDK.

The incompatible bytecode appears in the bar method in the following example, where an extra RuntimeVisibleTypeAnnotations(@LTest1678$Foo;() : CLASS_EXTENDS 0, 0;) is added. This CLASS_EXTENDS breaks Byte Buddy's runtime operations. Here's the corresponding example:

public class Test1678 {
    public static void main(String[] args) throws IOException {
        ClassFileLocator locator = ClassFileLocator.ForClassLoader.of(Test1678.class.getClassLoader());
        TypePool pool = TypePool.Default.of(locator);
        pool.describe(Test1678.class.getName()).resolve();
    }
    @Retention(RetentionPolicy.RUNTIME)
    @Target({ElementType.TYPE_USE, ElementType.TYPE_PARAMETER})
    public @interface Foo {
    }
    public static Object bar() {
        return new Iterable<@Foo Object>() {
            @Override
            public Iterator<Object> iterator() {
                return null;
            }
        };
    }
}

So far, I've only tested on the Windows platform. Java 9, 10, and 11 all failed, while Java 8, 12, 16, 17, 21, and 22 all worked successfully.

[raphw]

That should be consistent for all OSes. This also sounds intuitive since 9, 10 and 11 are unsupported versions with no bug fix backports.

[dogourd]

Are you planning to make Byte Buddy compatible with this specific value? From reviewing previous issues #1509 , it looks like there's some precedent for handling such cases.

From the code alone, my first instinct is that the segment return new Iterable<@Foo Object> is what's causing the javac error.

On one hand, being inside the method body, it doesn't seem like it should cause a type resolution error. On the other hand, the JVM doesn’t reject it at runtime, so it doesn’t appear to be a serious error from the JVM's perspective either.

[raphw]

I can certainly add it since it is an enduring compiler bug. Could you create a reproduction without dependencies for me to integrate in Byte Buddy?

[dogourd]

Of course! I can provide the source code and the corresponding bytecode files. Is that what you need?

Built from JDK-11. It includes the source file and bytecode file for net.bytebuddy.test.precompiled.v11.ClassExtendsTypeReference ClassExtendsTypeReference.zip

[raphw]

This is now handled and supported by a test. It will be released with the next version. Sorry for not taking care of this earlier.