APNS Connection Issue

[stager0909-ray]

hello

I'm using pushy version 0.15.1.

ApnsClientBuilder builder = new ApnsClientBuilder()
                .setApnsServer(apnsHost)
                .setMetricsListener(new MicrometerApnsClientMetricsListener(meterRegistry))
                .setConnectionTimeout(Duration.ofSeconds(3))
                .setIdlePingInterval(Duration.ofSeconds(60))
                .setGracefulShutdownTimeout(Duration.ofSeconds(60))
                .setConcurrentConnections(concurrentConnections)
                .setEventLoopGroup(apnsEventLoopGroup)
                .setSigningKey();

When there is an APNS issue, the following error occurs and the connection cannot be restored.

java.io.IOException: Stream closed before a reply was received
	at com.eatthepath.pushy.apns.ApnsClientHandler.<clinit>(ApnsClientHandler.java:84)
	at com.eatthepath.pushy.apns.TokenAuthenticationApnsClientHandler$TokenAuthenticationApnsClientHandlerBuilder.build(TokenAuthenticationApnsClientHandler.java:84)
	at com.eatthepath.pushy.apns.TokenAuthenticationApnsClientHandler$TokenAuthenticationApnsClientHandlerBuilder.build(TokenAuthenticationApnsClientHandler.java:56)
	at io.netty.handler.codec.http2.AbstractHttp2ConnectionHandlerBuilder.buildFromCodec(AbstractHttp2ConnectionHandlerBuilder.java:550)
	at io.netty.handler.codec.http2.AbstractHttp2ConnectionHandlerBuilder.buildFromConnection(AbstractHttp2ConnectionHandlerBuilder.java:539)
	at io.netty.handler.codec.http2.AbstractHttp2ConnectionHandlerBuilder.build(AbstractHttp2ConnectionHandlerBuilder.java:503)
	at com.eatthepath.pushy.apns.ApnsClientHandler$ApnsClientHandlerBuilder.build(ApnsClientHandler.java:145)
	at com.eatthepath.pushy.apns.ApnsChannelFactory$1.initChannel(ApnsChannelFactory.java:129)
	at com.eatthepath.pushy.apns.ApnsChannelFactory$1.initChannel(ApnsChannelFactory.java:97)
	at io.netty.channel.ChannelInitializer.initChannel(ChannelInitializer.java:129)
	at io.netty.channel.ChannelInitializer.handlerAdded(ChannelInitializer.java:112)
	at io.netty.channel.AbstractChannelHandlerContext.callHandlerAdded(AbstractChannelHandlerContext.java:938)
	at io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:609)
	at io.netty.channel.DefaultChannelPipeline.access$100(DefaultChannelPipeline.java:46)
	at io.netty.channel.DefaultChannelPipeline$PendingHandlerAddedTask.execute(DefaultChannelPipeline.java:1463)
	at io.netty.channel.DefaultChannelPipeline.callHandlerAddedForAllHandlers(DefaultChannelPipeline.java:1115)
	at io.netty.channel.DefaultChannelPipeline.invokeHandlerAddedIfNeeded(DefaultChannelPipeline.java:650)
	at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:514)
	at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:429)
	at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:486)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at java.lang.Thread.run(Thread.java:750)

Restarting the server solves the problem by making a new connection.

Connections are being monitored using MicrometerApnsClientMetricsListener.OPEN_CONNECTIONS_GAUGE_NAME .

As you can see from the metrics, when an APNS issue occurs, the connection goes down and is not restored. What should I check?

thank you

[jchambers]

…when an APNS issue occurs, the connection goes down and is not restored.

Can you elaborate on this point? Is there anything in the logs that says why the connection was closed (i.e. a GOAWAY frame from the server)?

[stager0909-ray]

hello

I am sending pushes using pushy in 2 idc environments.

A few days ago, the APNS OPEN Connection dropped to 0 in both environments at a specific time and the above error occurred. Since it occurred at the same time, I think the issue may have occurred on the APNS side.

One IDC returned to normal after some time, but the other was not restored.

Normalized IDC is connected through squid (proxy) when connecting to APNS. The unrecovered place is proxying through hardware.

I'm guessing it's a hardware proxy issue. However, if the 'Stream closed before a reply was received' error occurred because the OPEN Connection was dropped even in pushy, I would like to inquire about whether it can be restored by establishing a new connection.

thank you

[jchambers]

@stager0909-ray I'm very sorry to have left this issue alone for so long. It does sound like this was an issue with your specific network (thank you for sharing your findings!). Let's close this issue for now; if you have a similar problem in the future, please open a new discussion and we can revisit the problem.

Thanks!