replicate
When ctx is canceled, StreamPrediction will throw an error
If the ctx used by StreamPrediction is canceled, StreamPrediction will throw an error. The expectation is that StreamPrediction stops processing without throwing an error.
Hi @yan5xu. Thanks for reporting this. It was our intention for context cancellation to return without an error. Can you share what error is being produced?
Hi @yan5xu. Thanks for reporting this. It was our intention for context cancellation to return without an error. Can you share what error is being produced?
lol, below is the error msg I collected in the log
failed to send request: Get "https://streaming-api.svc.us.c.replicate.net/v1/streams/fge5z76wnqszkj5rqi3cxr4w6bwirlju6vxfvqxeoaiqnrij7sga": context canceled
@yan5xu Thank you for clarifying. If the error happens because the context is canceled for the original HTTP request for the stream, I think it's reasonable to bubble that up. And on reflection, I think it's less confusing if context errors are always returned, even when used to cancel a stream. Would you agree with that?
In the meantime, my recommendation would be to check for context cancellation errors and handle those differently:
if errors.Is(err, context.Canceled) {
// ...
}