Runtime class loading causes crash on 1.52.0; DD Java Agent clashes with OTel Agent JAR

[bvoelz19]

Tracer Version(s)

1.52.0

Java Version(s)

21.0.4

JVM Vendor

Other (please specify in comments)

Bug Report

Hi,

My JDK 21 application started crashing at runtime after pulling in the latest DD Java Agent JAR (v1.52.0). At runtime, I am using Groovy to dynamically load a set of classes, and I saw the following error:

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
General error during instruction selection: java.lang.NoClassDefFoundError: io/opentelemetry/javaagent/instrumentation/internal/classloader/BootstrapPackagesHelper
java.lang.RuntimeException: java.lang.NoClassDefFoundError: io/opentelemetry/javaagent/instrumentation/internal/classloader/BootstrapPackagesHelper
	at org.codehaus.groovy.control.CompilationUnit$IPrimaryClassNodeOperation.doPhaseOperation(CompilationUnit.java:972)
	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:692)
	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:666)
	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:365)
	at groovy.lang.GroovyClassLoader.lambda$parseClass$2(GroovyClassLoader.java:314)
	at org.codehaus.groovy.runtime.memoize.StampedCommonCache.compute(StampedCommonCache.java:163)
	at org.codehaus.groovy.runtime.memoize.StampedCommonCache.getAndPut(StampedCommonCache.java:154)
	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:314)
	at groovy.lang.GroovyShell.parseClass(GroovyShell.java:572)
	at groovy.lang.GroovyShell.parse(GroovyShell.java:585)
	at groovy.lang.GroovyShell.parse(GroovyShell.java:564)
	at groovy.lang.GroovyShell.parse(GroovyShell.java:552)
	at groovy.lang.GroovyShell.evaluate(GroovyShell.java:534)
        <REDACTED>
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:102)
	at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:64)
	at org.springframework.boot.loader.launch.JarLauncher.main(JarLauncher.java:40)
Caused by: java.lang.NoClassDefFoundError: io/opentelemetry/javaagent/instrumentation/internal/classloader/BootstrapPackagesHelper
	at groovy.lang.GroovyClassLoader$InnerLoader.loadClass(GroovyClassLoader.java:585)
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1027)
	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
	at groovy.lang.GroovyClassLoader.access$400(GroovyClassLoader.java:90)
	at groovy.lang.GroovyClassLoader$ClassCollector.createClass(GroovyClassLoader.java:682)
	at groovy.lang.GroovyClassLoader$ClassCollector.onClassNode(GroovyClassLoader.java:699)
	at groovy.lang.GroovyClassLoader$ClassCollector.call(GroovyClassLoader.java:704)
	at org.codehaus.groovy.control.CompilationUnit$3.call(CompilationUnit.java:806)
	at org.codehaus.groovy.control.CompilationUnit$IPrimaryClassNodeOperation.doPhaseOperation(CompilationUnit.java:938)
	... 21 more
Caused by: java.lang.ClassNotFoundException: io.opentelemetry.javaagent.instrumentation.internal.classloader.BootstrapPackagesHelper
	... 31 more

I am using both the DD Agent JAR as well as the OTel Agent JAR in my Application. The DD Agent is listed first in my Java Opts, and hence takes priority. This was not happening on earlier versions of the DD Agent JAR.

As a work-around, I have reverted to use v1.51.0 and the issue is resolved there. I may also consider trying to re-order the agents, such that the OTel Agent JAR takes precedence. However, curious what may have changed in the DD Agent JAR in this release, which would have caused this. Thanks!

Expected Behavior

Application starts up, with no issue loading classes at runtime.

Reproduction Code

			ClassLoader appClassLoader = MyApplication.class.getClassLoader();
			GroovyClassLoader groovyClassLoader = new GroovyClassLoader(appClassLoader);
			GroovyShell shell = new GroovyShell(groovyClassLoader, binding);

			PathMatchingResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
			Resource[] resources = resolver.getResources("classpath:*.groovy");
			Arrays.sort(resources, Comparator.comparing(Resource::getFilename));
			for (Resource resource: resources) {
				Reader reader = new InputStreamReader(resource.getInputStream());
				shell.evaluate(reader, resource.getFilename());
			}

And, ensure that the JVM JAVA_OPTS has both the OTel (latest) and DD Agent (1.52.0) JARs.

[mcculls]

Hi @bvoelz19 - in general we don't recommend attaching two tracing javaagents that will both inject advice into the same classes. This is because the injected advice might interact in unexpected ways.

We'll see if this can be fixed for the next release, but please be aware that this is not a recommended setup.

[mcculls]

Also is it possible to attach a more complete example which can be unzipped and run?

[bvoelz19]

Thanks for checking on it, @mcculls . Understood on the two-agent issue. Give me a few days and I will see if I can put together a sample application reproducing the issue.