kubeai
kubeai copied to clipboard
Configure a default timeout of 30 minutes
Any request that's pending for more than 30 minutes should return a 408 HTTP timout error
This timeout should be configurable. A value of 0 would disable it such that there is no timeout.
Any suggestions for where you would want the implementation to go, I was thinking maybe a timer could go here:
https://github.com/substratusai/lingo/blob/7a474239f2dd806a0f97fd9e453abf77461b8384/pkg/proxy/request.go#L38
I am not up to speed on the correct implementation off the cuff so I skimmed this article and it looks pretty informative: https://ieftimov.com/posts/make-resilient-golang-net-http-servers-using-timeouts-deadlines-context-cancellation/