continue
continue copied to clipboard
continuedev
Handle 5xx errors more gracefully in Chat
Before submitting your bug report
- [ ] I believe this is a bug. I'll try to join the Continue Discord for questions
- [ ] I'm not able to find an open issue that reports the same bug
- [ ] I've seen the troubleshooting guide on the Continue Docs
Relevant environment info
No response
Description
When we receive a 5xx error from a provider while streaming Chat responses, there's a few ways we could improve the experience:
- Make it clearer that the error is on the provider, not Continue, and recommend retrying the prompt
- Don't clear out the chat history (e.g. 90% of the message was streamed, last 10% hits a 5xx - current behavior is to clear the 90% that has already streamed)
We've had multiple users report 529 errors from Anthropic, so this issue has a relatively wide impact.
Original user feedback: https://discord.com/channels/1108621136150929458/1292863736100687923/1292863736100687923
Yeah. I personally don't mind the current notification or going into the logs, but it was entirely unclear that the overloaded issue was just the API failing.
However, my biggest suggestion would be in how the chat feature handles this type of error. The most frustrating part of this is how the completion completely disappears from the chat history. This is especially annoying when the completion is like 90% done, and I don't really even need the part that failed.
My ideal situation would be this: If a completion fails, Continue shows the error modal like it is, but with a more clear message indicating that Anthropic's API is what failed. Continue keeps the portion of the completion that was successful in the window until I click a "retry" or "continue anyways" button.
I am not really sensitive to pricing and I am not sure if Anthropic charges you for tokens in a failed completion, but I feel like some Continue customers could be irritated that they lost tokens they paid for if Continue wipes the completion on error.
Implementation
I think a good UX here could be to put the error notification directly into the Chat rather than a popup.
To reproduce
No response
Log output
No response
