jetty.project icon indicating copy to clipboard operation
jetty.project copied to clipboard

Published 20 hours ago • verified publisher icon jetty

java.lang.NullPointerException: Cannot invoke "org.eclipse.jetty.io.ArrayByteBufferPool$Buffer.acquire()" because "buffer" is null

Open brettkail-wk opened this issue 1 year ago • 1 comments

Jetty version(s) 12.0.6

Jetty Environment ee10

Java version/vendor (use: java -version)

openjdk version "21.0.2" 2024-01-16 LTS
OpenJDK Runtime Environment Corretto-21.0.2.14.1 (build 21.0.2+14-LTS)
OpenJDK 64-Bit Server VM Corretto-21.0.2.14.1 (build 21.0.2+14-LTS, mixed mode, sharing)

OS type/version Amazon Linux 2

Description Thread http-3782 logged a warning:

Task run failed
java.lang.NullPointerException: Cannot invoke "org.eclipse.jetty.io.ArrayByteBufferPool$Buffer.acquire()" because "buffer" is null
	at org.eclipse.jetty.io.ArrayByteBufferPool.acquire(ArrayByteBufferPool.java:226)
	at org.eclipse.jetty.server.internal.HttpConnection.fillRequestBuffer(HttpConnection.java:507)
	at org.eclipse.jetty.server.internal.HttpConnection.parseAndFillForContent(HttpConnection.java:497)
	at org.eclipse.jetty.server.internal.HttpConnection$HttpStreamOverHTTP1.read(HttpConnection.java:1344)
	at org.eclipse.jetty.server.HttpStream.consumeAvailable(HttpStream.java:117)
	at org.eclipse.jetty.server.internal.HttpConnection$HttpStreamOverHTTP1.consumeAvailable(HttpConnection.java:1110)
	at org.eclipse.jetty.server.HttpStream$Wrapper.consumeAvailable(HttpStream.java:215)
	at org.eclipse.jetty.server.internal.HttpChannelState.onFailure(HttpChannelState.java:459)
	at org.eclipse.jetty.server.internal.HttpChannelState$HttpChannelSerializedInvoker.onError(HttpChannelState.java:1768)
	at org.eclipse.jetty.util.thread.SerializedInvoker$Link.run(SerializedInvoker.java:197)
	at org.eclipse.jetty.server.internal.HttpConnection$DemandContentCallback.succeeded(HttpConnection.java:678)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:99)
	at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:478)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:441)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:293)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:201)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:410)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:971)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1201)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1156)
	at java.base/java.lang.Thread.run(Thread.java:1583)

~110ms later, thread http-3743 logging filter logged an info message for the start of a request. (See below.)

~60ms later, thread http-3782 logged another warning:

writeError: status=500, message=java.lang.NullPointerException: Cannot invoke "org.eclipse.jetty.io.ArrayByteBufferPool$Buffer.acquire()" because "buffer" is null, response=ErrorResponse@94f22d1{500,GET@3ae44aa <<REDACTED>> HTTP/1.1}
java.lang.NullPointerException: Cannot invoke "org.eclipse.jetty.io.ArrayByteBufferPool$Buffer.acquire()" because "buffer" is null
	at org.eclipse.jetty.io.ArrayByteBufferPool.acquire(ArrayByteBufferPool.java:226)
	at org.eclipse.jetty.server.internal.HttpConnection$SendCallback.process(HttpConnection.java:773)
	at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:250)
	at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:231)
	at org.eclipse.jetty.server.internal.HttpConnection$HttpStreamOverHTTP1.send(HttpConnection.java:1418)
	at org.eclipse.jetty.server.HttpStream$Wrapper.send(HttpStream.java:179)
	at org.eclipse.jetty.server.internal.HttpChannelState$ChannelResponse.write(HttpChannelState.java:1240)
	at org.eclipse.jetty.server.Response$Wrapper.write(Response.java:754)
	at org.eclipse.jetty.server.handler.ContextResponse.write(ContextResponse.java:56)
	at org.eclipse.jetty.ee10.servlet.ServletContextResponse.write(ServletContextResponse.java:288)
	at org.eclipse.jetty.ee10.servlet.HttpOutput.channelWrite(HttpOutput.java:204)
	at org.eclipse.jetty.ee10.servlet.HttpOutput.complete(HttpOutput.java:435)
	at org.eclipse.jetty.ee10.servlet.ServletContextResponse.completeOutput(ServletContextResponse.java:212)
	at org.eclipse.jetty.ee10.servlet.ServletChannel.handle(ServletChannel.java:562)
	at org.eclipse.jetty.ee10.servlet.ServletHandler.handle(ServletHandler.java:464)
	at org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:763)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:151)
	at org.eclipse.jetty.server.Server.handle(Server.java:179)
	at org.eclipse.jetty.server.internal.HttpChannelState$HandlerInvoker.run(HttpChannelState.java:619)
	at org.eclipse.jetty.server.internal.HttpConnection.onFillable(HttpConnection.java:410)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:322)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:99)
	at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:478)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:441)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:293)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:201)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:410)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:971)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1201)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1156)
	at java.base/java.lang.Thread.run(Thread.java:1583)
	Suppressed: java.lang.IllegalStateException: s=HANDLING rs=COMPLETED os=ABORTED is=IDLE awp=false se=false i=false al=0
		at org.eclipse.jetty.ee10.servlet.ServletChannelState.completed(ServletChannelState.java:1052)
		at org.eclipse.jetty.util.Callback$3.failed(Callback.java:168)
		at org.eclipse.jetty.ee10.servlet.HttpOutput.onWriteComplete(HttpOutput.java:252)
		at org.eclipse.jetty.ee10.servlet.HttpOutput$WriteCompleteCB.failed(HttpOutput.java:1760)
		at org.eclipse.jetty.ee10.servlet.HttpOutput$CompleteWriteCompleteCB.failed(HttpOutput.java:1777)
		at org.eclipse.jetty.server.handler.ContextHandler$ScopedContext.accept(ContextHandler.java:1179)
		at org.eclipse.jetty.server.handler.ContextResponse$1.failed(ContextResponse.java:47)
		at org.eclipse.jetty.server.internal.HttpChannelState$ChannelResponse.lambda$failed$3(HttpChannelState.java:1294)
		at org.eclipse.jetty.util.thread.SerializedInvoker$Link.run(SerializedInvoker.java:191)
		at org.eclipse.jetty.util.thread.SerializedInvoker.run(SerializedInvoker.java:117)
		at org.eclipse.jetty.server.internal.HttpChannelState$ChannelResponse.failed(HttpChannelState.java:1294)
		at org.eclipse.jetty.io.AbstractConnection.lambda$failedCallback$0(AbstractConnection.java:99)
		at org.eclipse.jetty.io.AbstractConnection.failedCallback(AbstractConnection.java:122)
		at org.eclipse.jetty.server.internal.HttpConnection.access$000(HttpConnection.java:83)
		at org.eclipse.jetty.server.internal.HttpConnection$SendCallback.onCompleteFailure(HttpConnection.java:913)
		at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:335)
		... 29 common frames omitted

~20ms, thread http-3743 logging filter logged an exception after the servlet had exited. Possibly related to the error above? Perhaps somehow a problem with a concurrent shared buffer?

java.lang.NullPointerException: Cannot invoke "org.eclipse.jetty.io.ArrayByteBufferPool$Buffer.acquire()" because "buffer" is null
	at org.eclipse.jetty.io.ArrayByteBufferPool.acquire(ArrayByteBufferPool.java:226)
	at org.eclipse.jetty.server.internal.HttpConnection$SendCallback.process(HttpConnection.java:773)
	at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:250)
	at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:231)
	at org.eclipse.jetty.server.internal.HttpConnection$HttpStreamOverHTTP1.send(HttpConnection.java:1418)
	at org.eclipse.jetty.server.HttpStream$Wrapper.send(HttpStream.java:179)
	at org.eclipse.jetty.server.internal.HttpChannelState$ChannelResponse.write(HttpChannelState.java:1240)
	at org.eclipse.jetty.server.Response$Wrapper.write(Response.java:754)
	at org.eclipse.jetty.server.handler.ContextResponse.write(ContextResponse.java:56)
	at org.eclipse.jetty.ee10.servlet.ServletContextResponse.write(ServletContextResponse.java:288)
	at org.eclipse.jetty.ee10.servlet.HttpOutput.channelWrite(HttpOutput.java:204)
	at org.eclipse.jetty.ee10.servlet.HttpOutput.close(HttpOutput.java:561)
	at java.base/java.util.Base64$EncOutputStream.close(Base64.java:1012)
	at <<REDACTED>>

How to reproduce? I don't know. We've only observed this sequence once. The server seemingly recovered, and the same threads successfully handled subsequent requests.

brettkail-wk avatar Apr 02 '24 20:04 brettkail-wk

The last NPE in ArrayByteBufferPool was ...

  • #11204

Addressed in release 12.0.6

joakime avatar Apr 02 '24 20:04 joakime