"Watcher stopped unexpectedly, will restart" warnings when using leader election

[clrxbl]

Describe the bug On Spring Boot 2.7.2 & Spring Cloud 2021.0.3 (with Spring Cloud Kubernetes 2.1.3), the following warning appears in the logs every now and then when Kubernetes leader election is in use:

2022-09-17 11:26:03.750  WARN 7 --- [/172.30.0.1/...] o.s.c.k.f.l.Fabric8LeaderRecordWatcher   : Watcher stopped unexpectedly, will restart

io.fabric8.kubernetes.client.WatcherException: too old resource version: 64484410 (64513614)
        at io.fabric8.kubernetes.client.dsl.internal.AbstractWatchManager.onStatus(AbstractWatchManager.java:273)
        at io.fabric8.kubernetes.client.dsl.internal.AbstractWatchManager.onMessage(AbstractWatchManager.java:257)
        at io.fabric8.kubernetes.client.dsl.internal.WatcherWebSocketListener.onMessage(WatcherWebSocketListener.java:93)
        at okhttp3.internal.ws.RealWebSocket.onReadMessage(RealWebSocket.kt:333)
        at okhttp3.internal.ws.WebSocketReader.readMessageFrame(WebSocketReader.kt:245)
        at okhttp3.internal.ws.WebSocketReader.processNextFrame(WebSocketReader.kt:106)
        at okhttp3.internal.ws.RealWebSocket.loopReader(RealWebSocket.kt:293)
        at okhttp3.internal.ws.RealWebSocket$connect$1.onResponse(RealWebSocket.kt:195)
        at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: too old resource version: 64484410 (64513614)
        ... 12 common frames omitted

This is seemingly harmless as the application functions just fine. It seems similar to #730

Sample N/A, simply running any application with leader election will return this warning after a while.

[ryanjbaxter]

Are you suggesting we don't print the warning?

[spring-cloud-issues]

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

[spring-cloud-issues]

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

[a-maggioni]

I'm using the latest version of spring-cloud-kubernetes-fabric8-leader and the warning is still there. According to this, Spring Cloud should migrate from using raw Watchers to Informers.

[wind57]

@a-maggioni you're right, we only switched to informers for discovery implementation, not for leader. Can you please create a defect so that we can track this properly?