Error when calling HEAD http://glu/console/rest/v1/.../plan/.../execution/...

[rantav]

It doesn't happen too often, certainly doesn't happen all the time, but every couple of days, maybe even weeks, I see this error in the API which I'm not sure what to make off it. When deployments run I send a HEAD call to get the status of the deployment every 5 seconds, for example: HEAD http://glu/console/rest/v1/outbrain/plan/af796de1-ad4a-4a87-8118- e91e5b6646b6/execution/1668 Now, most of the time this call succeeds and returns the up to date status of this deployment, sometimes, however, it fails with 500. I have the log from the client side but digging through glu's console log, I have nothing... There are quite a few other errors, which I'll report in another bug, but there's no such correlating log line at the same time that this error occurs (the API client and the console run on the same hosts, so clocks are in sync)

The error on the client side looks like this:

Glu returned an error for the request HEAD http://glu.outbrain.com:8080/console/rest/v1/outbrain/plan/af796de1-ad4a-4a87-8118- e91e5b Debug Data: HEAD status Status code: 500 Response headers: [Date: Sun, 07 Aug 2011 14:45:32 GMT , Set-Cookie: JSESSIONID=v6ps4ltv7qvr10fnf4b13rg4;Path=/console , Expires: Thu, 01-Jan-1970 00:00:00 GMT , Set-Cookie: rememberMe=deleteMe; Path=/console; Expires=Sat, 06-Aug-2011 14:45:32 GMT , Content-Language: en-US , Content-Type: text/html;charset=UTF-8 , Transfer-Encoding: chunked , Server: Jetty(7.2.2.v20101205) ] Response Text: null

Let me know if that's enough details for you or how can I gather more details. As mentioned, this doesn't happen a lot so it'll be hard to repro.

What's interesting is that the deployment was successful at last, no issues there

[ypujante]

Which version of the console are you using? Without an entry in the console log file that is going to be fairly hard to debug :(

I am wondering if anybody else has been experiencing the same problem.

[rantav]

The version is 3.2.0. Unfortunately, the error could not be correlated - it happened at 10:45 (from the API client's perspective) but the closest log entries I could find on the console were from 10:42 which I will paste below and another one from ... I don't know when... it doesn't have a time, I wonder why, maybe I'm reading this wrong. Anyhow, I'll past both server side logs here,

The one from 10:42, which is 3 minutes before the client got the error, so I assume it's not related (they use the same clock, they run on the same host) 2011-08-07 10:42:17.897:WARN::/console/plan/deployments java.lang.RuntimeException: org.springframework.web.util.NestedServletException: Request processing failed; nested exception is org.eclipse.jetty.io.RuntimeIOException: org.eclipse.jetty.io.EofExcept ion at com.opensymphony.sitemesh.webapp.decorator.BaseWebAppDecorator.render(BaseWebAppDecorator.java:40) at org.codehaus.groovy.grails.web.sitemesh.GrailsPageFilter.doFilter(GrailsPageFilter.java:143) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:359) at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:275) at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:343) at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:272) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:83) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.apache.shiro.grails.SavedRequestFilter.doFilter(SavedRequestFilter.java:55) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.codehaus.groovy.grails.web.servlet.mvc.GrailsWebRequestFilter.doFilterInternal(GrailsWebRequestFilter.java:69) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.codehaus.groovy.grails.web.filters.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:69) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:473) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:479) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:929) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:403) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:184) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:864) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:247) ...

And another one right after it which I'm not sure when it actually occured ... 532613603 [qtp2136389305-1852] ERROR org.codehaus.groovy.grails.web.errors.GrailsExceptionResolver - Exception occurred inside setter of org.linkedin.glu.console.domain.DbSystemModel.content org.hibernate.PropertyAccessException: Exception occurred inside setter of org.linkedin.glu.console.domain.DbSystemModel.content at org.linkedin.glu.console.domain.DbSystemModel.findCurrent(DbSystemModel.groovy:92) at org.linkedin.glu.console.domain.DbSystemModel$findCurrent$10.callStatic(Unknown Source) at org.linkedin.glu.console.domain.DbSystemModel.findCurrent(DbSystemModel.groovy:100) at org.linkedin.glu.console.domain.DbSystemModel$findCurrent$9.call(Unknown Source) at AuthFilters$__clinit__closure1_closure6_closure14.doCall(AuthFilters.groovy:134) at AuthFilters$__clinit__closure1_closure6_closure14.doCall(AuthFilters.groovy) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:534) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1351) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:473) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:516) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:929) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:403) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:184) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:864) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:285) at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:116) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:359) at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:275) at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:343) at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:272) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:83) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.apache.shiro.grails.SavedRequestFilter.doFilter(SavedRequestFilter.java:55) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) ... at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:590) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:212) at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:426) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:508) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.access$000(SelectChannelEndPoint.java:34) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:451) at java.lang.Thread.run(Thread.java:619) Caused by: java.lang.reflect.InvocationTargetException ... 60 more Caused by: groovy.lang.MissingMethodException: No signature of method: static org.linkedin.groovy.util.json.JsonUtils.toValue() is applicable for argument types: (java.lang.Integer) values: [40279] Possible solutions: toValue(java.lang.Object), toMap(org.json.JSONObject) at org.linkedin.groovy.util.json.JsonUtils$_toMap_closure1.doCall(JsonUtils.groovy:98) at org.linkedin.groovy.util.json.JsonUtils.toMap(JsonUtils.groovy:97) at org.linkedin.groovy.util.json.JsonUtils.toValue(JsonUtils.groovy:78) at org.linkedin.groovy.util.json.JsonUtils$_toMap_closure1.doCall(JsonUtils.groovy:98) at org.linkedin.groovy.util.json.JsonUtils.toMap(JsonUtils.groovy:97) at org.linkedin.groovy.util.json.JsonUtils.toValue(JsonUtils.groovy:78) at org.linkedin.groovy.util.json.JsonUtils$toValue.callStatic(Unknown Source) at org.linkedin.groovy.util.json.JsonUtils$toValue.callStatic(Unknown Source) at org.linkedin.groovy.util.json.JsonUtils$_toList_closure2.doCall(JsonUtils.groovy:114) at org.linkedin.groovy.util.json.JsonUtils.toList(JsonUtils.groovy:113) at org.linkedin.groovy.util.json.JsonUtils.toValue(JsonUtils.groovy:82) at org.linkedin.groovy.util.json.JsonUtils$_toMap_closure1.doCall(JsonUtils.groovy:98) at org.linkedin.groovy.util.json.JsonUtils.toMap(JsonUtils.groovy:97) at org.linkedin.groovy.util.json.JsonUtils.toValue(JsonUtils.groovy:78) at org.linkedin.groovy.util.json.JsonUtils$toValue.callStatic(Unknown Source) at org.linkedin.groovy.util.json.JsonUtils.fromJSON(JsonUtils.groovy:63) at org.linkedin.groovy.util.json.JsonUtils$fromJSON.call(Unknown Source) at org.linkedin.glu.provisioner.core.model.JSONSystemModelSerializer.deserialize(JSONSystemModelSerializer.groovy:34) at org.linkedin.glu.provisioner.core.model.SystemModelSerializer$deserialize$1.call(Unknown Source) at org.linkedin.glu.console.domain.DbSystemModel.setContent(DbSystemModel.groovy:57) ... 60 more

[rantav]

oh, wait wait wait! I see something which is in the console.log (but wasn't in the error log). This should help: What does EOF mean in this case? It looks like something low level... Sorry for being verbose, I'll just paste the whole chabang here...

2011/08/07 10:42:17.806 ERROR [GrailsExceptionResolver] org.eclipse.jetty.io.EofException org.eclipse.jetty.io.RuntimeIOException: org.eclipse.jetty.io.EofException at org.eclipse.jetty.io.UncheckedPrintWriter.setError(UncheckedPrintWriter.java:107) at org.eclipse.jetty.io.UncheckedPrintWriter.write(UncheckedPrintWriter.java:234) at tmp_jetty_0_0_0_0_8080_org_linkedin_glu_console_webapp_3_2_0_war__console_any__webapp_WEB_INF_grails_app_views_layouts_main_gsp.run(tmp_jetty_0_0_0_0_8080_org_linkedin_glu_console_webapp_3_2_0_w ar__console_any__webapp_WEB_INF_grails_app_views_layouts_main_gsp:190) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:534) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1351) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:473) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:516) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:929) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:403) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:184) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:864) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:285) at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:116) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:359) at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:275) at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:343) at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:272) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:83) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.apache.shiro.grails.SavedRequestFilter.doFilter(SavedRequestFilter.java:55) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:473) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:479) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:929) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:403) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:184) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:864) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:247) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:151) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:114) at org.eclipse.jetty.server.Server.handle(Server.java:352) at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:596) at org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1051) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:590) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:212) at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:426) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:508) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.access$000(SelectChannelEndPoint.java:34) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:451) at java.lang.Thread.run(Thread.java:619) Caused by: org.eclipse.jetty.io.EofException at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:911) at org.eclipse.jetty.http.AbstractGenerator.blockForOutput(AbstractGenerator.java:505) at org.eclipse.jetty.http.AbstractGenerator.flush(AbstractGenerator.java:436) at org.eclipse.jetty.server.HttpOutput.flush(HttpOutput.java:89) at org.eclipse.jetty.server.HttpConnection$Output.flush(HttpConnection.java:1142) at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:172) at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:96) at java.io.ByteArrayOutputStream.writeTo(ByteArrayOutputStream.java:109) at org.eclipse.jetty.server.HttpWriter.write(HttpWriter.java:258) at org.eclipse.jetty.io.UncheckedPrintWriter.write(UncheckedPrintWriter.java:225) ... 54 more Caused by: java.io.IOException: Broken pipe at org.eclipse.jetty.io.nio.ChannelEndPoint.flush(ChannelEndPoint.java:187) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.flush(SelectChannelEndPoint.java:275) at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:855) ... 63 more 2011/08/07 10:42:17.852 ERROR [GroovyPagesServlet] Error processing GSP: org.eclipse.jetty.io.EofException org.eclipse.jetty.io.RuntimeIOException: org.eclipse.jetty.io.EofException at org.eclipse.jetty.io.UncheckedPrintWriter.setError(UncheckedPrintWriter.java:107) at org.eclipse.jetty.io.UncheckedPrintWriter.write(UncheckedPrintWriter.java:234) at tmp_jetty_0_0_0_0_8080_org_linkedin_glu_console_webapp_3_2_0_war__console_any__webapp_WEB_INF_grails_app_views_layouts_main_gsp.run(tmp_jetty_0_0_0_0_8080_org_linkedin_glu_console_webapp_3_2_0_w ar__console_any__webapp_WEB_INF_grails_app_views_layouts_main_gsp:190) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:534) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1351) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:473) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:516) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:929) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:403) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:184) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:864) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:285) at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:116) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:359) at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:275) at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:343) at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:272) at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:83) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.apache.shiro.grails.SavedRequestFilter.doFilter(SavedRequestFilter.java:55) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1322) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:473) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:479) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:929) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:403) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:184) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:864) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:247) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:151) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:114) at org.eclipse.jetty.server.Server.handle(Server.java:352) at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:596) at org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1051) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:590) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:212) at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:426) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:508) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.access$000(SelectChannelEndPoint.java:34) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:451) at java.lang.Thread.run(Thread.java:619) Caused by: org.eclipse.jetty.io.EofException at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:911) at org.eclipse.jetty.http.AbstractGenerator.blockForOutput(AbstractGenerator.java:505) at org.eclipse.jetty.http.AbstractGenerator.flush(AbstractGenerator.java:436) at org.eclipse.jetty.server.HttpOutput.flush(HttpOutput.java:89) at org.eclipse.jetty.server.HttpConnection$Output.flush(HttpConnection.java:1142) at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:172) at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:96) at java.io.ByteArrayOutputStream.writeTo(ByteArrayOutputStream.java:109) at org.eclipse.jetty.server.HttpWriter.write(HttpWriter.java:258) at org.eclipse.jetty.io.UncheckedPrintWriter.write(UncheckedPrintWriter.java:225) ... 54 more Caused by: java.io.IOException: Broken pipe at org.eclipse.jetty.io.nio.ChannelEndPoint.flush(ChannelEndPoint.java:187) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.flush(SelectChannelEndPoint.java:275) at org.eclipse.jetty.http.HttpGenerator.flushBuffer(HttpGenerator.java:855) ... 63 more

[ypujante]

I have seen this kind of error in the past (not with glu though) and usually it happens because the client closes the connection before reading the response. I don't know if that could be that your client is somewhat misbehaving? Another potential candidate would be a bug in the jetty layer since it seems to be coming from there? I don't know if there is a more recent version of jetty but you may want to try with one if there is one, or with tomcat. I know that LinkedIn uses tomcat not jetty. I do not know if they have seen this error (worth asking on the forum).