OkGoDoIt
TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds
I often get this error. But other times it works without issues. Any ideas on how to change the timeout? Has anyone else had this issue?
System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
System.Threading.Tasks.TaskCanceledException
The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at OpenAI_API.EndpointBase.HttpRequestRaw(String url, HttpMethod verb, Object postData, Boolean streaming)
at OpenAI_API.EndpointBase.HttpRequest[T](String url, HttpMethod verb, Object postData)
at OpenAI_API.EndpointBase.HttpPost[T](String url, Object postData)
at OpenAI_API.Chat.ChatEndpoint.CreateChatCompletionAsync(ChatRequest request)
at OpenAI_API.Chat.Conversation.GetResponseFromChatbot()
at xxxx.xx.GetResponse(Conversation chat, String content) in C:\Code\personal\xxxx\xxxx\xxxx\xxx.cs:line 44
at Xunit.Sdk.TestInvoker`1.<>c__DisplayClass48_1.<<InvokeTestMethodAsync>b__1>d.MoveNext() in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\Runners\TestInvoker.cs:line 264
--- End of stack trace from previous location ---
at Xunit.Sdk.ExecutionTimer.AggregateAsync(Func`1 asyncAction) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\ExecutionTimer.cs:line 48
at Xunit.Sdk.ExceptionAggregator.RunAsync(Func`1 code) in C:\Dev\xunit\xunit\src\xunit.core\Sdk\ExceptionAggregator.cs:line 90
System.TimeoutException
The operation was canceled.
Exception doesn't have a stacktrace
System.Threading.Tasks.TaskCanceledException
The operation was canceled.
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
System.IO.IOException
Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter)
at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer)
at System.Net.Http.HttpConnection.InitialFillAsync(Boolean async)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
System.Net.Sockets.SocketException
The I/O operation has been aborted because of either a thread exit or an application request.
Exception doesn't have a stacktrace
+1
We need to accept CancellationToken on async functions and pass it to the HTTP requests.
a vote for making this user settable. I'm working with some big contexts in GPT-4 and they can take awhile. I have verified in OpenAI's Playground that in GPT-4 chat mode my prompts take well over 100 seconds to successfully complete.
I download the repository and made an ad-hoc fix, increasing the timeout to 1000 seconds. I rebuilt the nuget file and installed it locally. I tested it on my application and it works. At temporary fix to the problem. Much better would be a user parameter when creating a chat or other session.
in EndpointBase.cs:
protected HttpClient GetClient() { if (_Api.Auth?.ApiKey is null) { throw new AuthenticationException("You must provide API authentication. Please refer to https://github.com/OkGoDoIt/OpenAI-API-dotnet#authentication for details."); }
HttpClient client;
var clientFactory = _Api.HttpClientFactory;
if (clientFactory != null)
{
client = clientFactory.CreateClient();
}
else
{
client = new HttpClient();
}
client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", _Api.Auth.ApiKey);
// Further authentication-header used for Azure openAI service
client.DefaultRequestHeaders.Add("api-key", _Api.Auth.ApiKey);
client.DefaultRequestHeaders.Add("User-Agent", UserAgent);
// Jim Rutt increased timeout to 1000
client.Timeout = TimeSpan.FromSeconds(1000);
if (!string.IsNullOrEmpty(_Api.Auth.OpenAIOrganization)) client.DefaultRequestHeaders.Add("OpenAI-Organization", _Api.Auth.OpenAIOrganization);
return client;
}
Why not instantiate your own HttpClient with whatever timeout settings you want, and pass that into the existing service constructors?
Why not instantiate your own HttpClient with whatever timeout settings you want, and pass that into the existing service constructors?
Do you have an example of this?
Of what? chatGPT can show you how to instantiate an HttpClient and set its timeout settings. Then you pass it into the constructor that accepts an HttpClient reference
For people that are new to .NET like me, this is how I did it in C# (Warning, I have no idea what I'm doing and chatgpt did most of this work )
Add a reference to System.Net.Http to your project
Add the following lines in the right places in your project:
using System.Net.Http;
public class CustomHttpClientFactory : IHttpClientFactory
{
public HttpClient CreateClient(string name)
{
return new HttpClient
{
Timeout = TimeSpan.FromSeconds(300)
};
}
}
OpenAIAPI api = new OpenAIAPI(myApiKey)
{
HttpClientFactory = new CustomHttpClientFactory()
}
I have sort of the opposite issue. Some times the call hangs for 100 seconds and that's a long time if the answer is not coming. So I would need a timeout of maybe 10-15 seconds but only if it hasn't started answering yet. As soon as it is answering, then there should not be any timeout in my opinion.
I have sort of the opposite issue. Some times the call hangs for 100 seconds and that's a long time if the answer is not coming. So I would need a timeout of maybe 10-15 seconds but only if it hasn't started answering yet. As soon as it is answering, then there should not be any timeout in my opinion.
Is there a way to understand that CreateChatCompletionAsync writing an answer or just stuck? If so maybe we can keep the time with Stopwatch and cancel the request manually.
Perhaps use StreamCompletionEnumerableAsync instead, so that you'd know if it was in the process of responding, versus just waiting for anything at all to come back?
I'm not sure this is a bug to be fixed. The SDK allows you to pass in your own HttpClient with your own preferences for timeouts, etc: https://github.com/OkGoDoIt/OpenAI-API-dotnet/issues/102#issuecomment-1563708070
The custom HTTP client is the "right" way to do this, but I should probably add a helper to make it easy to set a custom timeout and take care of this behind the scenes. The custom HTTP client is a bit clunky.
