HttpInstrumenterVertxTracer error when DefaultHttp2Connection onStreamClosed
Describe the bug
2025-04-15 20:26:53,062 ERROR io.netty.handler.codec.http2.DefaultHttp2Connection - Caught Throwable from listener onStreamClosed.
java.lang.NullPointerException: Cannot invoke "io.opentelemetry.context.Context.get(io.opentelemetry.context.ContextKey)" because "context" is null
at io.opentelemetry.instrumentation.api.internal.HttpRouteState.fromContextOrNull(HttpRouteState.java:25)
at io.opentelemetry.instrumentation.api.semconv.http.HttpServerRoute.update(HttpServerRoute.java:99)
at io.quarkus.opentelemetry.runtime.tracing.intrumentation.vertx.HttpInstrumenterVertxTracer.sendResponse(HttpInstrumenterVertxTracer.java:103)
at io.quarkus.opentelemetry.runtime.tracing.intrumentation.vertx.HttpInstrumenterVertxTracer.sendResponse(HttpInstrumenterVertxTracer.java:50)
at io.quarkus.opentelemetry.runtime.tracing.intrumentation.vertx.OpenTelemetryVertxTracer.sendResponse(OpenTelemetryVertxTracer.java:47)
at io.quarkus.opentelemetry.runtime.tracing.intrumentation.vertx.OpenTelemetryVertxTracer.sendResponse(OpenTelemetryVertxTracer.java:16)
at io.quarkus.opentelemetry.runtime.tracing.intrumentation.vertx.OpenTelemetryVertxTracingFactory$VertxDelegator.sendResponse(OpenTelemetryVertxTracingFactory.java:78)
at io.vertx.core.http.impl.Http2ServerStream.onClose(Http2ServerStream.java:235)
at io.vertx.core.http.impl.Http2ConnectionBase.onStreamClosed(Http2ConnectionBase.java:157)
at io.vertx.core.http.impl.VertxHttp2ConnectionHandler$1.onStreamClosed(VertxHttp2ConnectionHandler.java:93)
at io.netty.handler.codec.http2.DefaultHttp2Connection.notifyClosed(DefaultHttp2Connection.java:355)
at io.netty.handler.codec.http2.DefaultHttp2Connection$ActiveStreams.removeFromActiveStreams(DefaultHttp2Connection.java:1034)
at io.netty.handler.codec.http2.DefaultHttp2Connection$ActiveStreams.deactivate(DefaultHttp2Connection.java:990)
at io.netty.handler.codec.http2.DefaultHttp2Connection$DefaultStream.close(DefaultHttp2Connection.java:515)
at io.netty.handler.codec.http2.DefaultHttp2Connection$DefaultStream.close(DefaultHttp2Connection.java:521)
at io.netty.handler.codec.http2.Http2ConnectionHandler.doCloseStream(Http2ConnectionHandler.java:919)
at io.netty.handler.codec.http2.Http2ConnectionHandler.access$900(Http2ConnectionHandler.java:64)
at io.netty.handler.codec.http2.Http2ConnectionHandler$2.operationComplete(Http2ConnectionHandler.java:632)
at io.netty.handler.codec.http2.Http2ConnectionHandler$2.operationComplete(Http2ConnectionHandler.java:629)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625)
at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105)
at io.netty.util.internal.PromiseNotificationUtil.trySuccess(PromiseNotificationUtil.java:48)
at io.netty.channel.DelegatingChannelPromiseNotifier.operationComplete(DelegatingChannelPromiseNotifier.java:52)
at io.netty.channel.DelegatingChannelPromiseNotifier.operationComplete(DelegatingChannelPromiseNotifier.java:31)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625)
at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105)
at io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84)
at io.netty.handler.codec.http2.Http2CodecUtil$SimpleChannelPromiseAggregator.tryPromise(Http2CodecUtil.java:380)
at io.netty.handler.codec.http2.Http2CodecUtil$SimpleChannelPromiseAggregator.trySuccess(Http2CodecUtil.java:347)
at io.netty.handler.codec.http2.Http2CodecUtil$SimpleChannelPromiseAggregator.trySuccess(Http2CodecUtil.java:259)
at io.netty.util.internal.PromiseNotificationUtil.trySuccess(PromiseNotificationUtil.java:48)
at io.netty.channel.ChannelOutboundBuffer.safeSuccess(ChannelOutboundBuffer.java:748)
at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:303)
at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:383)
at io.netty.channel.epoll.AbstractEpollStreamChannel.writeBytesMultiple(AbstractEpollStreamChannel.java:307)
at io.netty.channel.epoll.AbstractEpollStreamChannel.doWriteMultiple(AbstractEpollStreamChannel.java:512)
at io.netty.channel.epoll.AbstractEpollStreamChannel.doWrite(AbstractEpollStreamChannel.java:424)
at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:929)
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.flush0(AbstractEpollChannel.java:557)
at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:893)
at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1369)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:935)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:921)
at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:907)
at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:195)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:941)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:921)
at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:907)
at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:966)
at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:253)
at io.vertx.core.http.impl.VertxHttp2ConnectionHandler.checkFlush(VertxHttp2ConnectionHandler.java:247)
at io.vertx.core.http.impl.VertxHttp2ConnectionHandler.writeData(VertxHttp2ConnectionHandler.java:242)
at io.vertx.core.http.impl.VertxHttp2Stream.doWriteData(VertxHttp2Stream.java:247)
at io.vertx.core.http.impl.VertxHttp2Stream.lambda$writeData$7(VertxHttp2Stream.java:212)
at io.vertx.core.http.impl.VertxHttp2Stream.lambda$queueForWrite$8(VertxHttp2Stream.java:229)
at io.opentelemetry.javaagent.bootstrap.executors.ContextPropagatingRunnable.run(ContextPropagatingRunnable.java:37)
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:994)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)
Expected behavior
fix NPE
Actual behavior
NullPointerException
How to Reproduce?
- open http2
- use opentelemetry
- test it
Output of uname -a or ver
No response
Output of java -version
21
Quarkus version or git rev
3.15.1
Build tool (ie. output of mvnw --version or gradlew --version)
No response
Additional information
No response
/cc @brunobat (opentelemetry), @radcortez (opentelemetry)
Am I correct in assuming (based on the stacktrace) that you are using the OpenTelemetry agent?
Am I correct in assuming (based on the stacktrace) that you are using the OpenTelemetry agent?我是否正确地假设(基于堆栈跟踪)您正在使用 OpenTelemetry 代理?
yes,we always use OpenTelemetry agent,but when we use http2,it error
@naah69, So you are mixing the OpenTelemetry agent and the Quarkus OpenTelemetry extension?
You should only use one and the recommended way is the extension: https://quarkus.io/guides/opentelemetry
@naah69, So you are mixing the OpenTelemetry agent and the Quarkus OpenTelemetry extension?
You should only use one and the recommended way is the extension: https://quarkus.io/guides/opentelemetry
@brunobat
The reason that we use both of them is we has custom trace filter and we need some opentelemery class of vertx in extension.
In my option,quarkus.otel.sdk.disabled=true won't open extension.
Is this way mixing agent and extension?
@brunobat @geoand
Does quarkus OpenTelemetry extension(default,without other jars) support these libraries and frameworks in page and custom filter?
We use many libraries and frameworks in OpenTelemetry Instrumentation.If it supportes,I think that we can remove agent.
In my test,when I add opentelemetry-okhttp-3.0 to pom.xml and send request by okhttp,there is no traceparent in request header at the server side.
Does OpenTelemetry extension support standalone library instrumentation?
Given you seem to do something specific, it would immensely help if you could prepare a small reproducer with proper instructions on how to reproduce the problem.
@naah69 We support tracing on these extensions, by using quarkus-opentelemetry: https://quarkus.io/guides/opentelemetry-tracing#quarkus-extensions-using-opentelemetry
I'm not sure why you need so send requests with okhttp when you have the quarkus-rest-client: https://quarkus.io/guides/rest-client...
I second Guillaume's comment about the need for a reproducer.
@naah69 any luck with the reproducer?
Closing for the lack of a reproducer