Add ability to translate exceptions

[eyurtsev]

trafficstars

Expose exception translation capability to allow users to provide their own logic for translating server side errors into client side errors

https://github.com/langchain-ai/langserve/blob/e1cec832da29173df89bfe50b35be8ce59b1a8b5/langserve/api_handler.py#L1214-L1214

Discussed in https://github.com/langchain-ai/langserve/discussions/473

^{Originally posted by yw791772 February 21, 2024} Hi, I'm following the astream method in the langserve client code, and I have a try catch around the async sse code. For testing purpose, I use an api key that has no quota, and I can see the 429 errors from the langserve app logs. But when I looking at the decoded message from it on the client side, I only see 500 with internal server error.

Any guidance on how I can surface the 429 errors on the client side, so I can build specific around this error

[yw791772]

2 of the most common use cases I encountered:

• 429 rate limiting
• context too large, which could turn out to be hard to detect, for example: all I see is a 400 bad request from open ai

[eyurtsev]

https://github.com/langchain-ai/langserve/discussions/545

[philbo]

@eyurtsev do you have any ideas about how to approach implementing this? I can try to contribute a solution

[ognjenAct]

hii guys, any updates on this topic? or possible workaround?

[danielatchobanikova]

Hi, I am experiencing the same issue. Is there any workaround?

We are using Bedrock with on-demand throughput which often returns Throttling Exceptions with 429 error code and we need to return the error code to the UI, so an appropriate message can be displayed to the end user.

[philbo]

I ended up working around it by implementing a custom APIHandler, then catching any exceptions raised by it and returning them in a basic JSON response with a 5xx status which the client can handle separately from the event stream.