Agent exits due to ConcurrentModficiationException

[graemerocher]

Occasionally I see the dreaded Spring loaded agent exiting error which is caused by:

| Error Caused by: java.util.ConcurrentModificationException | Error at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) | Error at java.util.AbstractList$Itr.next(AbstractList.java:343) | Error at org.springsource.loaded.TypeRegistry.isReloadableTypeName(TypeRegistry.java:695) | Error at org.springsource.loaded.agent.SpringLoadedPreProcessor.preProcess(SpringLoadedPreProcessor.java:160) | Error at org.springsource.loaded.agent.ClassPreProcessorAgentAdapter.transform(ClassPreProcessorAgentAdapter.java:102) | Error ... 187 more

[aclement]

I added a possible fix for this, always a little tricky when I don't have a reliable testcase.

[posto]

I noticed the same error in Spring Loaded 1.1.3 (running Grails 2.2.3, Linux, JDK 1.7.21).

| Error java.lang.RuntimeException: Reloading agent exited via exception, please raise a jira
| Error     at org.springsource.loaded.agent.ClassPreProcessorAgentAdapter.transform(ClassPreProcessorAgentAdapter.java:104)
| Error     at sun.instrument.TransformerManager.transform(TransformerManager.java:188)
| Error     at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:424)
| Error     at java.lang.ClassLoader.defineClass1(Native Method)
| Error     at java.lang.ClassLoader.defineClass(ClassLoader.java:792)
| Error     at org.codehaus.groovy.reflection.ClassLoaderForClassArtifacts.define(ClassLoaderForClassArtifacts.java:42)
| Error     at org.codehaus.groovy.reflection.ClassLoaderForClassArtifacts$1.run(ClassLoaderForClassArtifacts.java:86)
| Error     at org.codehaus.groovy.reflection.ClassLoaderForClassArtifacts$1.run(ClassLoaderForClassArtifacts.java:84)
| Error     at java.security.AccessController.doPrivileged(Native Method)
| Error     at org.codehaus.groovy.reflection.ClassLoaderForClassArtifacts.defineClassAndGetConstructor(ClassLoaderForClassArtifacts.java:84)
| Error     at org.codehaus.groovy.runtime.callsite.CallSiteGenerator.compileStaticMethod(CallSiteGenerator.java:241)
| Error     at org.codehaus.groovy.reflection.CachedMethod.createStaticMetaMethodSite(CachedMethod.java:285)
| Error     at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.createStaticMetaMethodSite(StaticMetaMethodSite.java:111)
| Error     at groovy.lang.MetaClassImpl.createStaticSite(MetaClassImpl.java:3028)
| Error     at groovy.lang.ExpandoMetaClass.createStaticSite(ExpandoMetaClass.java:1296)
...
| Error     at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)
| Error     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
| Error     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
| Error     at java.lang.Thread.run(Thread.java:724)
| Error Caused by: java.util.ConcurrentModificationException
| Error     at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:819)
| Error     at java.util.ArrayList$Itr.next(ArrayList.java:791)
| Error     at org.springsource.loaded.SystemPropertyConfiguredIsReloadableTypePlugin.shouldBeMadeReloadable(SystemPropertyConfiguredIsReloadableTypePlugin.java:124)
| Error     at org.springsource.loaded.TypeRegistry.isReloadableTypeName(TypeRegistry.java:699)
| Error     at org.springsource.loaded.agent.SpringLoadedPreProcessor.preProcess(SpringLoadedPreProcessor.java:160)
| Error     at org.springsource.loaded.agent.ClassPreProcessorAgentAdapter.transform(ClassPreProcessorAgentAdapter.java:102)
| Error     ... 166 more

[ilx]

I tried 1.1.3 on websphere 8.5 and got similar stacktrace:

[10.09.13. 15:03:49:215 CEST] 00000069 SpringLoadedP E org.springsource.loaded.agent.SpringLoadedPreProcessor preProcess Unexpected problem transforming call sites
                                 java.util.ConcurrentModificationException
    at java.util.AbstractList$SimpleListIterator.next(AbstractList.java:64)
    at org.springsource.loaded.TypeRegistry.couldBeReloadable(TypeRegistry.java:610)
    at org.springsource.loaded.TypeRegistry.isReloadableTypeName(TypeRegistry.java:710)
    at org.springsource.loaded.TypeRegistry.isReloadableTypeName(TypeRegistry.java:657)
    at org.springsource.loaded.MethodInvokerRewriter$RewriteClassAdaptor$RewritingMethodAdapter.visitFieldInsn(MethodInvokerRewriter.java:766)
    at sl.org.objectweb.asm.ClassReader.accept(Unknown Source)
    at sl.org.objectweb.asm.ClassReader.accept(Unknown Source)
    at org.springsource.loaded.MethodInvokerRewriter.rewrite(MethodInvokerRewriter.java:330)
    at org.springsource.loaded.MethodInvokerRewriter.rewrite(MethodInvokerRewriter.java:96)
    at org.springsource.loaded.TypeRegistry.methodCallRewriteUseCacheIfAvailable(TypeRegistry.java:793)
    at org.springsource.loaded.agent.SpringLoadedPreProcessor.preProcess(SpringLoadedPreProcessor.java:264)
    at org.springsource.loaded.agent.ClassPreProcessorAgentAdapter.transform(ClassPreProcessorAgentAdapter.java:102)
    at sun.instrument.TransformerManager.transform(TransformerManager.java:181)
    at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:377)
    at java.lang.ClassLoader.defineClassImpl(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:262)
    at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.defineClass(DefaultClassLoader.java:188)
    at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.defineClass(ClasspathManager.java:580)
    at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findClassImpl(ClasspathManager.java:550)
    at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClassImpl(ClasspathManager.java:481)
    at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClass_LockClassName(ClasspathManager.java:460)
    at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClass(ClasspathManager.java:447)
    at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.findLocalClass(DefaultClassLoader.java:216)
    at org.eclipse.osgi.internal.loader.BundleLoader.findLocalClass(BundleLoader.java:393)
    at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:469)
    at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:422)
    at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:410)
    at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:107)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:627)
    at com.ibm.ws.naming.jbatch.javaCNBatchContextImpl.<init>(javaCNBatchContextImpl.java:92)
    at com.ibm.ws.naming.java.javaCNContextImpl.getBatchModeContext(javaCNContextImpl.java:133)
    at com.ibm.ws.ejbcontainer.runtime.AbstractEJBRuntime.getBatchContext(AbstractEJBRuntime.java:1447)
    at com.ibm.ws.ejbcontainer.runtime.AbstractEJBRuntime.startModule(AbstractEJBRuntime.java:728)
    at com.ibm.ws.runtime.component.EJBContainerImpl.start(EJBContainerImpl.java:3591)
    at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:1170)
    at com.ibm.ws.runtime.component.DeployedApplicationImpl.fireDeployedObjectStart(DeployedApplicationImpl.java:1370)
    at com.ibm.ws.runtime.component.DeployedModuleImpl.start(DeployedModuleImpl.java:638)
    at com.ibm.ws.runtime.component.DeployedApplicationImpl.start(DeployedApplicationImpl.java:968)
    at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:769)
    at com.ibm.ws.runtime.component.CompositionUnitMgrImpl$CUInitializer.run(CompositionUnitMgrImpl.java:998)
    at com.ibm.wsspi.runtime.component.WsComponentImpl$_AsynchInitializer.run(WsComponentImpl.java:349)
    at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1783)

[davydotcom]

I can make it happen every time in grails 3.0.2 with handlebars-asset-pipeline installed and a few hbs files. very odd

[aclement]

If you can attach a failing sample (or email me one) I can dig into it. I suspect it is related to the package name cache we keep which records which packages we are currently considering reloadable.