coai icon indicating copy to clipboard operation
coai copied to clipboard

Implement Timeout Configuration for Response and Stream Waiting

Open xgen20 opened this issue 10 months ago • 1 comments

I suggest implementing the ability to configure the timeout for waiting for a response and waiting for a response stream. Because in my case, it's not always enough to receive a response from GPT-4 (chat to API version). The response sometimes takes 2+ minutes to write, and the Сhat Nio stops processing the stream after 1-2 minutes. And if the API doesn't send anything at all, it resets the request after 20+ seconds and makes a new attempt. I would like to have the ability to adjust this parameter for each channel, as well as the parameter for the maximum waiting time for the stream.

xgen20 avatar Apr 24 '24 03:04 xgen20

good suggestion, consider implementing

zmh-program avatar Apr 26 '24 06:04 zmh-program

The Chat Nio Pro is already supported, waiting for @Sh1n3zZ to release the open source version.

Support Setting Http Timeout: set env MAX_TIMEOUT=30 to set http timeout to 30s.

https://github.com/Deeptrain-Community/chatnio-pro/commit/0bec4a78ec275fb9abd52ac87b38e44a5aae086d

zmh-program avatar Jun 27 '24 13:06 zmh-program

Hi,

Completed.

Regards, Junhai Deng

Sh1n3zZ avatar Jun 29 '24 18:06 Sh1n3zZ