Marshalling error in IndexRequestQueue.executeRequests()

[msmolyak]

Our Grails application that uses the Elastic Search Grails plugin occasionally throws the following exception during the execution of JMeter tests with large number of users (upwards of 500). The error seems to indicate that the ServiceItem object (the only root level object in our app) is not associated with a Hibernate session and thus cannot be converted to JSON due to the fact that some of its attributes are lazily instantiated. Why could this happen?

Thank you,

Michael

org.hibernate.LazyInitializationException: could not initialize proxy - no Session
    at org.hibernate.proxy.AbstractLazyInitializer.initialize(AbstractLazyInitializer.java:86)
    at org.codehaus.groovy.grails.orm.hibernate.proxy.HibernateProxyHandler.unwrapProxy(HibernateProxyHandler.java:83)
    at org.codehaus.groovy.grails.orm.hibernate.cfg.GrailsHibernateUtil.unwrapProxy(GrailsHibernateUtil.java:404)
    at org.codehaus.groovy.grails.orm.hibernate.cfg.GrailsHibernateUtil$unwrapProxy.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:40)
    at org.codehaus.groovy.grails.orm.hibernate.cfg.GrailsHibernateUtil$unwrapProxy.call(Unknown Source)
    at org.codehaus.groovy.grails.plugins.orm.hibernate.HibernatePluginSupport$__clinit__closure94.doCall(HibernatePluginSupport.groovy:397)
    at sun.reflect.GeneratedMethodAccessor460.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:266)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:51)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:149)
    at org.codehaus.groovy.grails.plugins.orm.hibernate.HibernatePluginSupport$__clinit__closure94.call(HibernatePluginSupport.groovy)
    at sun.reflect.GeneratedMethodAccessor459.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1058)
    at groovy.lang.ExpandoMetaClass.invokeMethod(ExpandoMetaClass.java:1070)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:886)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:930)
    at groovy.lang.ExpandoMetaClass.invokeMethod(ExpandoMetaClass.java:1070)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:886)
    at org.codehaus.groovy.runtime.InvokerHelper.invokePogoMethod(InvokerHelper.java:793)
    at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:776)
    at org.codehaus.groovy.runtime.metaclass.ClosureMetaMethod$1.invoke(ClosureMetaMethod.java:137)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
    at groovy.lang.MetaClassImpl.getProperty(MetaClassImpl.java:1602)
    at groovy.lang.ExpandoMetaClass.getProperty(ExpandoMetaClass.java:1099)
    at groovy.lang.MetaClassImpl.getProperty(MetaClassImpl.java:3311)
    at groovy.lang.ExpandoMetaClass.getProperty(ExpandoMetaClass.java:1111)
    at marketplace.FieldValue.getProperty(FieldValue.groovy)
    at org.codehaus.groovy.runtime.InvokerHelper.getProperty(InvokerHelper.java:156)
    at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.getProperty(ScriptBytecodeAdapter.java:455)
    at org.grails.plugins.elasticsearch.conversion.marshall.DeepDomainClassMarshaller.doMarshall(DeepDomainClassMarshaller.groovy:23)
    at sun.reflect.GeneratedMethodAccessor644.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:266)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:51)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:44)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:56)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:149)
    at org.grails.plugins.elasticsearch.conversion.marshall.DefaultMarshaller.marshall(DefaultMarshaller.groovy:34)
    at org.grails.plugins.elasticsearch.conversion.marshall.Marshaller$marshall.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:40)
    at org.grails.plugins.elasticsearch.conversion.marshall.Marshaller$marshall.call(Unknown Source)
    at org.grails.plugins.elasticsearch.conversion.JSONDomainFactory.delegateMarshalling(JSONDomainFactory.groovy:123)
    at org.grails.plugins.elasticsearch.conversion.JSONDomainFactory$delegateMarshalling.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:40)
    at org.grails.plugins.elasticsearch.conversion.JSONDomainFactory$delegateMarshalling.call(Unknown Source)
    at org.grails.plugins.elasticsearch.conversion.marshall.DefaultMarshallingContext.delegateMarshalling(DefaultMarshallingContext.groovy:75)
    at org.grails.plugins.elasticsearch.conversion.marshall.DefaultMarshallingContext$delegateMarshalling$0.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:40)
    at org.grails.plugins.elasticsearch.conversion.marshall.DefaultMarshallingContext$delegateMarshalling$0.call(Unknown Source)
    at org.grails.plugins.elasticsearch.conversion.marshall.DeepDomainClassMarshaller.doMarshall(DeepDomainClassMarshaller.groovy:31)
    at sun.reflect.GeneratedMethodAccessor644.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:266)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:51)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:44)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:56)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:149)
    at org.grails.plugins.elasticsearch.conversion.marshall.DefaultMarshaller.marshall(DefaultMarshaller.groovy:34)
    at org.grails.plugins.elasticsearch.conversion.marshall.Marshaller$marshall.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:40)
    at org.grails.plugins.elasticsearch.conversion.marshall.Marshaller$marshall.call(Unknown Source)
    at org.grails.plugins.elasticsearch.conversion.JSONDomainFactory.delegateMarshalling(JSONDomainFactory.groovy:123)
    at org.grails.plugins.elasticsearch.conversion.JSONDomainFactory$delegateMarshalling.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:40)
    at org.grails.plugins.elasticsearch.conversion.JSONDomainFactory$delegateMarshalling.call(Unknown Source)
    at org.grails.plugins.elasticsearch.conversion.marshall.DefaultMarshallingContext.delegateMarshalling(DefaultMarshallingContext.groovy:75)
    at sun.reflect.GeneratedMethodAccessor479.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSite.invoke(PogoMetaMethodSite.java:225)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:51)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:153)
    at org.grails.plugins.elasticsearch.conversion.marshall.DefaultMarshallingContext.delegateMarshalling(DefaultMarshallingContext.groovy)
    at org.grails.plugins.elasticsearch.conversion.marshall.DefaultMarshallingContext$delegateMarshalling.call(Unknown Source)
    at org.grails.plugins.elasticsearch.conversion.marshall.CollectionMarshaller$_doMarshall_closure1.doCall(CollectionMarshaller.groovy:9)
    at sun.reflect.GeneratedMethodAccessor648.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1058)
    at groovy.lang.ExpandoMetaClass.invokeMethod(ExpandoMetaClass.java:1070)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:886)
    at groovy.lang.Closure.call(Closure.java:282)
    at groovy.lang.Closure.call(Closure.java:295)
    at org.codehaus.groovy.runtime.DefaultGroovyMethods.collect(DefaultGroovyMethods.java:1770)
    at org.codehaus.groovy.runtime.DefaultGroovyMethods.collect(DefaultGroovyMethods.java:1754)
    at org.codehaus.groovy.runtime.dgm$71.invoke(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:270)
    at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:52)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:40)
    at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:54)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:124)
    at org.grails.plugins.elasticsearch.conversion.marshall.CollectionMarshaller.doMarshall(CollectionMarshaller.groovy:8)
    at sun.reflect.GeneratedMethodAccessor789.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:266)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:51)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:44)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:56)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:149)
    at org.grails.plugins.elasticsearch.conversion.marshall.DefaultMarshaller.marshall(DefaultMarshaller.groovy:34)
    at org.grails.plugins.elasticsearch.conversion.marshall.Marshaller$marshall.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:40)
    at org.grails.plugins.elasticsearch.conversion.marshall.Marshaller$marshall.call(Unknown Source)
    at org.grails.plugins.elasticsearch.conversion.JSONDomainFactory.delegateMarshalling(JSONDomainFactory.groovy:123)
    at sun.reflect.GeneratedMethodAccessor477.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSite.invoke(PogoMetaMethodSite.java:225)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:51)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:157)
    at org.grails.plugins.elasticsearch.conversion.JSONDomainFactory.delegateMarshalling(JSONDomainFactory.groovy)
    at sun.reflect.GeneratedMethodAccessor476.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1058)
    at groovy.lang.ExpandoMetaClass.invokeMethod(ExpandoMetaClass.java:1070)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:886)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1003)
    at groovy.lang.ExpandoMetaClass.invokeMethod(ExpandoMetaClass.java:1070)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:886)
    at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.callCurrent(PogoMetaClassSite.java:66)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:153)
    at org.grails.plugins.elasticsearch.conversion.JSONDomainFactory$_buildJSON_closure3.doCall(JSONDomainFactory.groovy:146)
    at sun.reflect.GeneratedMethodAccessor475.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1058)
    at groovy.lang.ExpandoMetaClass.invokeMethod(ExpandoMetaClass.java:1070)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:886)
    at groovy.lang.Closure.call(Closure.java:282)
    at groovy.lang.Closure.call(Closure.java:295)
    at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1220)
    at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1196)
    at org.codehaus.groovy.runtime.dgm$110.invoke(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:270)
    at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:52)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:124)
    at org.grails.plugins.elasticsearch.conversion.JSONDomainFactory.buildJSON(JSONDomainFactory.groovy:144)
    at org.grails.plugins.elasticsearch.index.IndexRequestQueue.toJSON(IndexRequestQueue.java:126)
    at org.grails.plugins.elasticsearch.index.IndexRequestQueue.executeRequests(IndexRequestQueue.java:180)
    at org.grails.plugins.elasticsearch.index.IndexRequestQueue$executeRequests.call(Unknown Source)
    at org.grails.plugins.elasticsearch.AuditEventListener.onFlush(AuditEventListener.groovy:178)
    at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1027)
    at org.springframework.orm.hibernate3.HibernateAccessor.flushIfNecessary(HibernateAccessor.java:390)
    at org.codehaus.groovy.grails.orm.hibernate.support.GrailsOpenSessionInViewInterceptor.flushIfNecessary(GrailsOpenSessionInViewInterceptor.java:123)
    at org.springframework.orm.hibernate3.support.OpenSessionInViewInterceptor.postHandle(OpenSessionInViewInterceptor.java:181)
    at org.codehaus.groovy.grails.orm.hibernate.support.GrailsOpenSessionInViewInterceptor.postHandle(GrailsOpenSessionInViewInterceptor.java:73)
    at org.springframework.web.servlet.handler.WebRequestHandlerInterceptorAdapter.postHandle(WebRequestHandlerInterceptorAdapter.java:61)
    at org.codehaus.groovy.grails.web.servlet.GrailsDispatcherServlet.doDispatch(GrailsDispatcherServlet.java:303)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:549)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:722) 

[msmolyak]

I am trying to understand the code from IndexRequestQueue.executeRequests() below. This question is related to the exception I posted above.

The way I understand it, the code obtains a Hibernate session for the current thread and then tries to reattach each entity to be indexed to that session. The question I have is: is it possible that the 'toIndex' list contains entities from different threads and thus different Hibernate sessions? Is it possible that when the executeRequests() method is called (as a result of a Hibernate session being flushed), the toIndex list would contain entities placed there by other threads and thus belonging to other Hibernate sessions? If this happens, the check

 if (session.contains(entity)) 

will return false and the entity will be treated as a transient one. In our case that causes a LazyInstantiationException since such unattached entity cannot obtain some of its attributes.

Michael

...
       for (Map.Entry<IndexEntityKey, Object> entry : toIndex.entrySet()) {
            SearchableClassMapping scm = elasticSearchContextHolder.getMappingContextByType(entry.getKey().getClazz());
            persistenceInterceptor.init();
            try {
                Session session = SessionFactoryUtils.getSession(sessionFactory, true);
                Object entity = entry.getValue();

                // If this not a transient instance, reattach it to the session
                if (session.contains(entity)) {
                    session.lock(entity, LockMode.NONE);
                    LOG.debug("Reattached entity to session");
                }

                XContentBuilder json = toJSON(entity);

                bulkRequestBuilder.add(
                        elasticSearchClient.prepareIndex()
                                .setIndex(scm.getIndexName())
                                .setType(scm.getElasticTypeName())
                                .setId(entry.getKey().getId()) // TODO : Composite key ?
                                .setSource(json)
                );
...

[abstratt]

I am seeing a similar issue, but no need for multiple users, our test suite is single threaded. Any workarounds?

Caused by LazyInitializationException: could not initialize proxy - no Session
->> 146 | doCall                 in org.grails.plugins.elasticsearch.conversion.JSONDomainFactory$_buildJSON_closure3
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
|   144 | buildJSON              in org.grails.plugins.elasticsearch.conversion.JSONDomainFactory
|   113 | toJSON . . . . . . . . in org.grails.plugins.elasticsearch.index.IndexRequestQueue
|   173 | executeRequests        in     ''
|   178 | onFlush . . . . . . .  in org.grails.plugins.elasticsearch.AuditEventListener
|   108 | doCall                 in org.grails.datastore.gorm.GormStaticApi$_methodMissing_closure2
|    18 | findTenantByServerName in com.mycorp.TenantService
|    17 | doFilter               in com.mycorp.util.MultiTenantFilters
|   270 | doFilter . . . . . . . in com.planetj.servlet.filter.compression.CompressingFilter
|   886 | runTask                in java.util.concurrent.ThreadPoolExecutor$Worker
|   908 | run . . . . . . . . .  in     ''
^   662 | run                    in java.lang.Thread

[zhuravskiy]

+1