csharp

csharp copied to clipboard

Describe the bug

OnError callback of Watcher<T>() reports the following error after ~40 minutes when watching for a Kubernetes resource for which no instance exists in the cluster and no new one is created during the runtime. After the error watcher is closed (OnClosed callback is invoked).

System.Net.Http.HttpRequestException: Error while copying content to a stream.
 ---> System.IO.EndOfStreamException: Attempted to read past the end of the stream.
   at k8s.LineSeparatedHttpContent.PeekableStreamReader.PeekLineAsync()
   at k8s.LineSeparatedHttpContent.SerializeToStreamAsync(Stream stream, TransportContext context)
   at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)
   at System.Net.Http.HttpContent.WaitAndReturnAsync[TState,TResult\](Task waitTask, TState state, Func`2 returnFunc)
   at k8s.Kubernetes.CreateResultAsync[T\](HttpRequestMessage httpRequest, HttpResponseMessage httpResponse, Nullable`1 watch, CancellationToken cancellationToken)
   at k8s.AbstractKubernetes.ICustomObjectsOperations_ListClusterCustomObjectWithHttpMessagesAsync[T\](String group, String version, String plural, Nullable`1 allowWatchBookmarks, String continueParameter, String fieldSelector, String labelSelector, Nullable`1 limit, String resourceVersion, String resourceVersionMatch, Nullable`1 timeoutSeconds, Nullable`1 watch, Nullable`1 pretty, IReadOnlyDictionary`2 customHeaders, CancellationToken cancellationToken)
   at k8s.AbstractKubernetes.k8s.ICustomObjectsOperations.ListClusterCustomObjectWithHttpMessagesAsync(String group, String version, String plural, Nullable`1 allowWatchBookmarks, String continueParameter, String fieldSelector, String labelSelector, Nullable`1 limit, String resourceVersion, String resourceVersionMatch, Nullable`1 timeoutSeconds, Nullable`1 watch, Nullable`1 pretty, IReadOnlyDictionary`2 customHeaders, CancellationToken cancellationToken)
   at k8s.WatcherExt.<>c__DisplayClass1_0`2.<<MakeStreamReaderCreator>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at k8s.Watcher`1.<>c.<CreateWatchEventEnumerator>b__21_1[TR\](Task`1 t)
   at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at k8s.Watcher`1.CreateWatchEventEnumerator(Func`1 streamReaderCreator, Action`1 onError, CancellationToken cancellationToken)+MoveNext()
   at k8s.Watcher`1.CreateWatchEventEnumerator(Func`1 streamReaderCreator, Action`1 onError, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at k8s.Watcher`1.WatcherLoop(CancellationToken cancellationToken)
   at k8s.Watcher`1.WatcherLoop(CancellationToken cancellationToken)

Kubernetes C# SDK Client Version 13.0.1

Server Kubernetes Version 1.26.10

Dotnet Runtime Version net7.0

To Reproduce After reading description of #828 I think that Kubernetes API server must close watch connection.

Expected behavior Even if connection is closed by Kubernetes API server, the watcher should be able to recover and no attempt should be made to read past end of stream.

KubeConfig N/A

Where do you run your app with Kubernetes SDK (please complete the following information):

• OS: Debian 11
• Environment: container running in Kubernetes cluster
• Azure, AKS

Additional context N/A