Can not automatic recovery after the watch mechanism is interrupted by etcd server reboot

[vinglogn]

# Issue I build the jetcd watch example with the version 0.5.3 and run it，but I got the wrong thing with the following error message，when the target etcd server is down。 It can not automatic recover after I resume the etcd server。

when I use etcdctl to test the watch function，It can automic recover when the etcd server is interrupted for a few minutes.

# Log 15:19:43.817|ERROR|WatchImpl - scheduled resume failed java.lang.IllegalStateException: call was half-closed at io.etcd.jetcd.shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:511) ~[jetcd-watch-0.5.3.jar:?] at io.etcd.jetcd.shaded.io.grpc.internal.ClientCallImpl.sendMessageInternal(ClientCallImpl.java:529) ~[jetcd-watch-0.5.3.jar:?] at io.etcd.jetcd.shaded.io.grpc.internal.ClientCallImpl.sendMessage(ClientCallImpl.java:520) ~[jetcd-watch-0.5.3.jar:?] at io.etcd.jetcd.shaded.io.grpc.ForwardingClientCall.sendMessage(ForwardingClientCall.java:37) ~[jetcd-watch-0.5.3.jar:?] at io.etcd.jetcd.shaded.io.grpc.stub.ClientCalls$CallToStreamObserverAdapter.onNext(ClientCalls.java:346) ~[jetcd-watch-0.5.3.jar:?] at io.etcd.jetcd.WatchImpl$WatcherImpl.resume(WatchImpl.java:151) ~[jetcd-watch-0.5.3.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[?:1.8.0_171] at io.etcd.jetcd.shaded.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) ~[jetcd-watch-0.5.3.jar:?]

    at io.etcd.jetcd.shaded.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69) ~[jetcd-watch-0.5.3.jar:?]
    at io.etcd.jetcd.shaded.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) ~[jetcd-watch-0.5.3.jar:?]
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [?:1.8.0_171]
    at java.util.concurrent.FutureTask.run(Unknown Source) [?:1.8.0_171]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source) [?:1.8.0_171]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) [?:1.8.0_171]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:1.8.0_171]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:1.8.0_171]
    at java.lang.Thread.run(Unknown Source) [?:1.8.0_171]

八月 18, 2020 3:19:44 下午 io.etcd.jetcd.shaded.io.grpc.internal.SerializingExecutor run 严重: Exception while executing runnable io.etcd.jetcd.shaded.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed@6e36be39 java.lang.NullPointerException at io.etcd.jetcd.WatchImpl$WatcherImpl.onError(WatchImpl.java:269) at io.etcd.jetcd.shaded.io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:449) at io.etcd.jetcd.shaded.io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39) at io.etcd.jetcd.shaded.io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23) at io.etcd.jetcd.shaded.io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40) at io.etcd.jetcd.ClientConnectionManager$AuthTokenInterceptor$1$1.onClose(ClientConnectionManager.java:336) at io.etcd.jetcd.shaded.io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:426) at io.etcd.jetcd.shaded.io.grpc.internal.ClientCallImpl.access$500(ClientCallImpl.java:66) at io.etcd.jetcd.shaded.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:689) at io.etcd.jetcd.shaded.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$900(ClientCallImpl.java:577) at io.etcd.jetcd.shaded.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:751) at io.etcd.jetcd.shaded.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:740) at io.etcd.jetcd.shaded.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) at io.etcd.jetcd.shaded.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)

[lburgazzoli]

Seems related to https://github.com/etcd-io/jetcd/issues/532

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[lburgazzoli]

@vinglogn can you please test against 0.5.4 ?

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days.