bug: Custom exception using FastAPI mount does not work as expected

[Amitg1]

Describe the bug

Hello,

Currently, the Bento REST API handles inference issues, such as IO errors or GPU out-of-memory errors, by returning a 500 internal server error to the user.

Upon seeking advice in the BentoML Slack community, it was recommended to integrate FastAPI to gain complete control over the exceptions returned. After implementing this suggestion, I observed that it functions as intended when executing "bentoml serve --development." However, when the "--development" flag is removed, the application reverts to responding with only a 500 internal server error instead of the custom error I specified.

To reproduce

bentoml serve --development --debug

bentoml serve --api-workers 1 --debug

Expected behavior

return custom expectations on prod mode

Environment

bentoml version 1.1.10 python 3.9.18