ServletChannelState#onError is not robust enough in updating state towards registered listeners
Jetty version(s) 12.0.30
Jetty Environment ee10
HTTP version HTTP 1.0, 1.1
Java version/vendor (use: java -version)
Does not matter:
java version "25.0.1" 2025-10-21 LTS
Java(TM) SE Runtime Environment (build 25.0.1+8-LTS-27)
Java HotSpot(TM) 64-Bit Server VM (build 25.0.1+8-LTS-27, mixed mode, sharing)
OS type/version Alpine distro-less linux alpine-minirootfs-3.22.2-x86
Description We have one automated test that simulates an unexpected crash of the application to test the shutdown consistency of the application. Recently we have noticed that we get an unexpected 200 response codes during this shutdown. Further analysis has showed that it is caused by an upgrade of jersey from 3.1.9 to 3.1.11. This was already reported to jersey: https://github.com/eclipse-ee4j/jersey/issues/6044. I suggest that you read that ticket as well.
After logs and source code analysis, we see that the following situation is happening:
org.eclipse.jetty.ee10.servlet.ServletChannelState#onError is called due to a simulated exception
This method delegates the call to asyncListeners:
runInContext(asyncEvent, () -> { for (AsyncListener listener : asyncListeners) { try { listener.onError(asyncEvent); }
This is heppening before sendError is invoked. And most importantly, before _sendError is set to true.
Now, as it was mentioned, the problem is coming from Jersey 3.1.11 and more precisely, from a new CompletedAsyncContextListener that was added there. The problem is that this listener is calling complete from the onError method: ` @Override public void onError(AsyncEvent event) throws IOException { complete(); }
@Override public void complete() { completed.set(true);
final AsyncContext asyncContext = asyncContextRef.getAndSet(null);
if (asyncContext != null) {
asyncContext.complete();
}
}
`
Which is in fact, delegated back to org.eclipse.jetty.ee10.servlet.ServletChannelState.complete. Now, since _sendError and other things from sendEror were not called yet, this will cause a response with default 200 status code to be sent, which is not correct. However, this can be fixed by making onError more robust by changing the order of calls: First call sendError method and only after it is done, delegate calls to asyncListeners listener.onError(asyncEvent);.
Logs when unexptected 200 is send:
[2025-12-09 11:26:15,730] [DEBUG] [] [o.e.j.u.component.AbstractLifeCycle] [] - STOPPING ServerConnector@1435a8ca{HTTP/1.1, (http/1.1)}{0.0.0.0:8080} [2025-12-09 11:26:15,731] [DEBUG] [] [o.e.j.u.component.AbstractLifeCycle] [] - STOPPING ServerConnectorManager@69780093[keys=8] [2025-12-09 11:26:15,731] [DEBUG] [] [o.e.j.u.component.AbstractLifeCycle] [] - STOPPING oeji.ManagedSelector@38280b42{STARTED}[id=9 keys=0 selected=0 updates=0 selection:tot=0/avg=0.00/max=0] [2025-12-09 11:26:15,734] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - Queued change lazy=false org.eclipse.jetty.io.ManagedSelector$CloseConnections@3ffdb7d7 on oeji.ManagedSelector@38280b42{STOPPING}[id=9 keys=0 selected=0 updates=0 selection:tot=0/avg=0.00/max=0] [2025-12-09 11:26:15,735] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - Wakeup on submit oeji.ManagedSelector@38280b42{STOPPING}[id=9 keys=0 selected=0 updates=1 selection:tot=0/avg=0.00/max=0] [2025-12-09 11:26:15,735] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - Selector sun.nio.ch.EPollSelectorImpl@2e94f495 woken with none selected [2025-12-09 11:26:15,735] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - Selector sun.nio.ch.EPollSelectorImpl@2e94f495 woken up from select, 0/0/0 selected [2025-12-09 11:26:15,735] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - Selector sun.nio.ch.EPollSelectorImpl@2e94f495 processing 0 keys, 1 updates [2025-12-09 11:26:15,735] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - updateable 1 [2025-12-09 11:26:15,735] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - update org.eclipse.jetty.io.ManagedSelector$CloseConnections@3ffdb7d7 [2025-12-09 11:26:15,735] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - Closing 0 connections on oeji.ManagedSelector@38280b42{STOPPING}[id=9 keys=0 selected=0 updates=0 selection:tot=0/avg=0.00/max=0] [2025-12-09 11:26:15,735] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - updates 0 [2025-12-09 11:26:15,735] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - Selector sun.nio.ch.EPollSelectorImpl@2e94f495 waiting with 0 keys [2025-12-09 11:26:15,737] [DEBUG] [] [o.e.j.util.thread.QueuedThreadPool] [] - ran qtp975358276-181-acceptor-0@347544b7-ServerConnector@1435a8ca{HTTP/1.1, (http/1.1)}{0.0.0.0:8080} in QueuedThreadPool[qtp975358276]@3a22c944{STARTED,8<=36<=200,i=3,r=-1,t=44335ms,q=0}[ReservedThreadExecutor@4dfe14d4{capacity=32,threads=ThreadIdPool@7e3062f0{capacity=32}}] [2025-12-09 11:26:15,737] [DEBUG] [] [o.e.j.util.thread.QueuedThreadPool] [] - Evict check, period=60000ms QueuedThreadPool[qtp975358276]@3a22c944{STARTED,8<=36<=200,i=4,r=-1,t=44335ms,q=0}[ReservedThreadExecutor@4dfe14d4{capacity=32,threads=ThreadIdPool@7e3062f0{capacity=32}}] [2025-12-09 11:26:15,737] [DEBUG] [] [o.e.j.util.thread.QueuedThreadPool] [] - Evict skipped, threshold=104335ms in the future QueuedThreadPool[qtp975358276]@3a22c944{STARTED,8<=36<=200,i=4,r=-1,t=44335ms,q=0}[ReservedThreadExecutor@4dfe14d4{capacity=32,threads=ThreadIdPool@7e3062f0{capacity=32}}] [2025-12-09 11:26:15,738] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - Queued change lazy=false org.eclipse.jetty.io.ManagedSelector$StopSelector@3b995478 on oeji.ManagedSelector@38280b42{STOPPING}[id=9 keys=0 selected=0 updates=0 selection:tot=0/avg=0.00/max=0] [2025-12-09 11:26:15,738] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - Wakeup on submit oeji.ManagedSelector@38280b42{STOPPING}[id=9 keys=0 selected=0 updates=1 selection:tot=0/avg=0.00/max=0] [2025-12-09 11:26:15,738] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - Selector sun.nio.ch.EPollSelectorImpl@2e94f495 woken with none selected [2025-12-09 11:26:15,738] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - Selector sun.nio.ch.EPollSelectorImpl@2e94f495 woken up from select, 0/0/0 selected [2025-12-09 11:26:15,738] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - Selector sun.nio.ch.EPollSelectorImpl@2e94f495 processing 0 keys, 1 updates [2025-12-09 11:26:15,738] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - updateable 1 [2025-12-09 11:26:15,738] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - update org.eclipse.jetty.io.ManagedSelector$StopSelector@3b995478 [2025-12-09 11:26:15,738] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - Closing sun.nio.ch.EPollSelectorImpl@2e94f495 on oeji.ManagedSelector@38280b42{STOPPING}[id=9 keys=-1 selected=-1 updates=0 selection:tot=0/avg=0.00/max=0] [2025-12-09 11:26:15,738] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - updates 0 [2025-12-09 11:26:15,738] [DEBUG] [] [o.e.j.util.thread.QueuedThreadPool] [] - ran org.eclipse.jetty.io.ManagedSelector$$Lambda/0x00000000a0946008@5316255e in QueuedThreadPool[qtp975358276]@3a22c944{STARTED,8<=36<=200,i=4,r=-1,t=44334ms,q=0}[ReservedThreadExecutor@4dfe14d4{capacity=32,threads=ThreadIdPool@7e3062f0{capacity=32}}] [2025-12-09 11:26:15,738] [DEBUG] [] [o.e.j.util.thread.QueuedThreadPool] [] - Evict check, period=60000ms QueuedThreadPool[qtp975358276]@3a22c944{STARTED,8<=36<=200,i=5,r=-1,t=44334ms,q=0}[ReservedThreadExecutor@4dfe14d4{capacity=32,threads=ThreadIdPool@7e3062f0{capacity=32}}] [2025-12-09 11:26:15,738] [DEBUG] [] [o.e.j.util.thread.QueuedThreadPool] [] - Evict skipped, threshold=104334ms in the future QueuedThreadPool[qtp975358276]@3a22c944{STARTED,8<=36<=200,i=5,r=-1,t=44334ms,q=0}[ReservedThreadExecutor@4dfe14d4{capacity=32,threads=ThreadIdPool@7e3062f0{capacity=32}}] [2025-12-09 11:26:15,738] [DEBUG] [] [o.e.j.u.component.AbstractLifeCycle] [] - STOPPING oejuts.AdaptiveExecutionStrategy@cd041f9/SelectorProducer@be694c0/IDLE/QueuedThreadPool[qtp975358276]@3a22c944{STARTED,8<=36<=200,i=5,r=-1,t=44334ms,q=0}[ReservedThreadExecutor@4dfe14d4{capacity=32,threads=ThreadIdPool@7e3062f0{capacity=32}}][pc=0,pic=0,pec=0,epc=0/0] [2025-12-09 11:26:15,738] [DEBUG] [] [o.e.j.u.component.AbstractLifeCycle] [] - STOPPED oejuts.AdaptiveExecutionStrategy@cd041f9/SelectorProducer@be694c0/IDLE/QueuedThreadPool[qtp975358276]@3a22c944{STARTED,8<=36<=200,i=5,r=-1,t=44334ms,q=0}[ReservedThreadExecutor@4dfe14d4{capacity=32,threads=ThreadIdPool@7e3062f0{capacity=32}}][pc=0,pic=0,pec=0,epc=0/0] [2025-12-09 11:26:15,738] [DEBUG] [] [o.e.j.u.component.AbstractLifeCycle] [] - STOPPED oeji.ManagedSelector@38280b42{STOPPED}[id=9 keys=-1 selected=-1 updates=0 selection:tot=0/avg=0.00/max=0] [2025-12-09 11:26:15,738] [DEBUG] [] [o.e.j.u.component.AbstractLifeCycle] [] - STOPPING oeji.ManagedSelector@e71bd92{STARTED}[id=8 keys=1 selected=0 updates=0 selection:tot=1/avg=1.00/max=1] [2025-12-09 11:26:15,738] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - Queued change lazy=false org.eclipse.jetty.io.ManagedSelector$CloseConnections@7a3a29dc on oeji.ManagedSelector@e71bd92{STOPPING}[id=8 keys=1 selected=0 updates=0 selection:tot=1/avg=1.00/max=1] [2025-12-09 11:26:15,738] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - Wakeup on submit oeji.ManagedSelector@e71bd92{STOPPING}[id=8 keys=1 selected=0 updates=1 selection:tot=1/avg=1.00/max=1] [2025-12-09 11:26:15,739] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - Selector sun.nio.ch.EPollSelectorImpl@369dbf21 woken with none selected [2025-12-09 11:26:15,739] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - Selector sun.nio.ch.EPollSelectorImpl@369dbf21 woken up from select, 0/0/1 selected [2025-12-09 11:26:15,739] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - Selector sun.nio.ch.EPollSelectorImpl@369dbf21 processing 0 keys, 1 updates [2025-12-09 11:26:15,739] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - updateable 1 [2025-12-09 11:26:15,739] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - update org.eclipse.jetty.io.ManagedSelector$CloseConnections@7a3a29dc [2025-12-09 11:26:15,739] [DEBUG] [] [o.eclipse.jetty.io.ManagedSelector] [] - Closing 1 connections on oeji.ManagedSelector@e71bd92{STOPPING}[id=8 keys=1 selected=0 updates=0 selection:tot=1/avg=1.00/max=1] [2025-12-09 11:26:15,739] [DEBUG] [] [o.e.j.s.internal.HttpChannelState] [] - onClose HttpChannelState@5143e81b{handling=null, handled=true, send=SENDING, completed=false, request=POST@22588103 http://localhost:8080/test-product/v1/persistence/scenarios/persist_up_to HTTP/1.1} stream=org.eclipse.jetty.server.internal.CompletionStreamWrapper@6268f42a [2025-12-09 11:26:15,741] [DEBUG] [] [o.e.j.s.internal.HttpChannelState] [] - onFailure HttpChannelState@5143e81b{handling=null, handled=true, send=SENDING, completed=false, request=POST@22588103 http://localhost:8080/test-product/v1/persistence/scenarios/persist_up_to HTTP/1.1} org.eclipse.jetty.io.EofException: null at org.eclipse.jetty.server.internal.HttpChannelState.onClose(HttpChannelState.java:525) at org.eclipse.jetty.server.internal.HttpConnection.close(HttpConnection.java:628) at org.eclipse.jetty.util.IO.close(IO.java:615) at org.eclipse.jetty.util.IO.close(IO.java:630) at org.eclipse.jetty.io.ManagedSelector$CloseConnections.update(ManagedSelector.java:1000) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.processUpdates(ManagedSelector.java:577) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:544) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:526) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:251) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:204) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:311) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:981) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1211) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1166) at java.base/java.lang.Thread.run(Thread.java:1474) [2025-12-09 11:26:15,743] [DEBUG] [] [o.e.j.util.thread.SerializedInvoker] [] - Offering task null, skipping it in HttpChannelSerializedInvoker@124e3398{name=HttpChannelState_readInvoker,tail=null,invoker=null} [2025-12-09 11:26:15,744] [DEBUG] [] [o.e.j.util.thread.SerializedInvoker] [] - Offering task null, skipping it in HttpChannelSerializedInvoker@ebab8b7{name=HttpChannelState_writeInvoker,tail=null,invoker=null} [2025-12-09 11:26:15,744] [DEBUG] [] [o.e.j.util.thread.SerializedInvoker] [] - Offering link Link@6e4ee70a{Queued by qtp975358276-577 at org.eclipse.jetty.server.internal.HttpChannelState.onFailure(HttpChannelState.java:500)[BLOCKING] -> null} of HttpChannelSerializedInvoker@124e3398{name=HttpChannelState_readInvoker,tail=null,invoker=null} [2025-12-09 11:26:15,744] [DEBUG] [] [o.e.j.util.thread.SerializedInvoker] [] - Running link Link@6e4ee70a{Queued by qtp975358276-577 at org.eclipse.jetty.server.internal.HttpChannelState.onFailure(HttpChannelState.java:500)[BLOCKING] -> null} of HttpChannelSerializedInvoker@124e3398{name=HttpChannelState_readInvoker,tail=Link@6e4ee70a{Queued by qtp975358276-577 at org.eclipse.jetty.server.internal.HttpChannelState.onFailure(HttpChannelState.java:500)[BLOCKING] -> null},invoker=null} [2025-12-09 11:26:15,744] [DEBUG] [] [o.e.j.s.internal.HttpChannelState] [] - invokeListeners HttpChannelState@5143e81b{handling=null, handled=true, send=SENDING, completed=false, request=POST@22588103 http://localhost:8080/test-product/v1/persistence/scenarios/persist_up_to HTTP/1.1} org.eclipse.jetty.server.handler.ContextRequest$$Lambda/0x00000000a1104380@55798ed8 org.eclipse.jetty.io.EofException: null at org.eclipse.jetty.server.internal.HttpChannelState.onClose(HttpChannelState.java:525) at org.eclipse.jetty.server.internal.HttpConnection.close(HttpConnection.java:628) at org.eclipse.jetty.util.IO.close(IO.java:615) at org.eclipse.jetty.util.IO.close(IO.java:630) at org.eclipse.jetty.io.ManagedSelector$CloseConnections.update(ManagedSelector.java:1000) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.processUpdates(ManagedSelector.java:577) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:544) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:526) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:251) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:204) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:311) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:981) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1211) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1166) at java.base/java.lang.Thread.run(Thread.java:1474) [2025-12-09 11:26:15,744] [DEBUG] [] [o.e.j.e.servlet.ServletChannelState] [] - asyncError ServletChannelState@1948a2f6{s=WAITING rs=ASYNC os=OPEN is=IDLE awp=false se=false i=false al=1} org.eclipse.jetty.io.EofException: null at org.eclipse.jetty.server.internal.HttpChannelState.onClose(HttpChannelState.java:525) at org.eclipse.jetty.server.internal.HttpConnection.close(HttpConnection.java:628) at org.eclipse.jetty.util.IO.close(IO.java:615) at org.eclipse.jetty.util.IO.close(IO.java:630) at org.eclipse.jetty.io.ManagedSelector$CloseConnections.update(ManagedSelector.java:1000) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.processUpdates(ManagedSelector.java:577) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:544) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:526) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:251) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:204) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:311) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:981) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1211) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1166) at java.base/java.lang.Thread.run(Thread.java:1474) [2025-12-09 11:26:15,745] [DEBUG] [] [o.e.j.ee10.servlet.ServletChannel] [] - handle http://localhost:8080/test-product/v1/persistence/scenarios/persist_up_to ServletChannel@41850adb{s=ServletChannelState@1948a2f6{s=WOKEN rs=ASYNC os=OPEN is=IDLE awp=false se=false i=false al=1},r=0,c=false/false,a=WOKEN,uri=http://localhost:8080/test-product/v1/persistence/scenarios/persist_up_to,age=1131} [2025-12-09 11:26:15,745] [DEBUG] [] [o.e.j.e.servlet.ServletChannelState] [] - handling ServletChannelState@1948a2f6{s=WOKEN rs=ASYNC os=OPEN is=IDLE awp=false se=false i=false al=1} [2025-12-09 11:26:15,745] [DEBUG] [] [o.e.j.ee10.servlet.ServletChannel] [] - action ASYNC_ERROR ServletChannel@41850adb{s=ServletChannelState@1948a2f6{s=HANDLING rs=ASYNC os=OPEN is=IDLE awp=false se=false i=false al=1},r=0,c=false/false,a=HANDLING,uri=http://localhost:8080/test-product/v1/persistence/scenarios/persist_up_to,age=1131} [2025-12-09 11:26:15,745] [DEBUG] [] [o.e.j.ee10.servlet.ServletChannel] [] - /test-product/v1/persistence/scenarios/persist_up_to org.eclipse.jetty.io.EofException: null at org.eclipse.jetty.server.internal.HttpChannelState.onClose(HttpChannelState.java:525) at org.eclipse.jetty.server.internal.HttpConnection.close(HttpConnection.java:628) at org.eclipse.jetty.util.IO.close(IO.java:615) at org.eclipse.jetty.util.IO.close(IO.java:630) at org.eclipse.jetty.io.ManagedSelector$CloseConnections.update(ManagedSelector.java:1000) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.processUpdates(ManagedSelector.java:577) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:544) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:526) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:251) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:204) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:311) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:981) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1211) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1166) at java.base/java.lang.Thread.run(Thread.java:1474) [2025-12-09 11:26:15,746] [DEBUG] [] [o.e.j.e.servlet.ServletChannelState] [] - onError s=HANDLING rs=ASYNC os=OPEN is=IDLE awp=false se=false i=false al=1 org.eclipse.jetty.io.EofException: null at org.eclipse.jetty.server.internal.HttpChannelState.onClose(HttpChannelState.java:525) at org.eclipse.jetty.server.internal.HttpConnection.close(HttpConnection.java:628) at org.eclipse.jetty.util.IO.close(IO.java:615) at org.eclipse.jetty.util.IO.close(IO.java:630) at org.eclipse.jetty.io.ManagedSelector$CloseConnections.update(ManagedSelector.java:1000) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.processUpdates(ManagedSelector.java:577) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:544) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:526) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:251) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:204) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:311) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:981) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1211) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1166) at java.base/java.lang.Thread.run(Thread.java:1474) [2025-12-09 11:26:15,747] [DEBUG] [] [o.e.j.e.servlet.ServletChannelState] [] - complete ServletChannelState@1948a2f6{s=HANDLING rs=ASYNC os=OPEN is=IDLE awp=false se=false i=false al=1} [2025-12-09 11:26:15,747] [DEBUG] [] [o.e.j.e.servlet.ServletChannelState] [] - unhandle ServletChannelState@1948a2f6{s=HANDLING rs=COMPLETE os=OPEN is=IDLE awp=false se=false i=false al=1} [2025-12-09 11:26:15,747] [DEBUG] [] [o.e.j.e.servlet.ServletChannelState] [] - nextAction(false) COMPLETE ServletChannelState@1948a2f6{s=HANDLING rs=COMPLETING os=OPEN is=IDLE awp=false se=false i=false al=1} [2025-12-09 11:26:15,747] [DEBUG] [] [o.e.j.ee10.servlet.ServletChannel] [] - action COMPLETE ServletChannel@41850adb{s=ServletChannelState@1948a2f6{s=HANDLING rs=COMPLETING os=OPEN is=IDLE awp=false se=false i=false al=1},r=0,c=false/false,a=HANDLING,uri=http://localhost:8080/test-product/v1/persistence/scenarios/persist_up_to,age=1133} [2025-12-09 11:26:15,747] [DEBUG] [] [o.e.jetty.ee10.servlet.HttpOutput] [] - complete(Callback@74831062{NON_BLOCKING, org.eclipse.jetty.ee10.servlet.ServletChannel$$Lambda/0x00000000a11239f8@6d954f64,org.eclipse.jetty.ee10.servlet.ServletChannel$$Lambda/0x00000000a1123e58@1e06679f}) s=CLOSING,api=BLOCKED,sc=false,e=null s=false e=null, c=HeapByteBuffer@3b90258d[p=0,l=0,c=0,r=0]={<<<>>>} [2025-12-09 11:26:15,747] [DEBUG] [] [o.e.j.server.handler.EventsHandler] [] - onResponseBegin of @1582a761{POST@22588103 http://localhost:8080/test-product/v1/persistence/scenarios/persist_up_to HTTP/1.1} status=0 headers=Vary: Accept-Encoding Set-Cookie: rememberMe=deleteMe; Path=/; Max-Age=0; Expires=Mon, 08-Dec-2025 11:26:14 GMT; SameSite=lax [2025-12-09 11:26:15,747] [DEBUG] [] [o.e.j.s.internal.ResponseHttpFields] [] - org.eclipse.jetty.server.internal.ResponseHttpFields@796ea5b3 committed