inference
inference copied to clipboard
Improve API structure + put non-async handlers properly
Search before asking
- [X] I have searched the Inference issues and found no similar bug report.
Bug
The HTTP API module requires re-design. It is large and contains plenty of handlers. Some handlers (running non-asyncio code) are marked async improperly, potentially making the API less performant (fastapi handles both async and non-async handlers, just stamps need to be placed properly). We have this mistake by the virtue of error handling wrapper which is async-first.
Please re-design API as maintenance activity.
Environment
No response
Minimal Reproducible Example
No response
Additional
No response
Are you willing to submit a PR?
- [ ] Yes I'd like to help by submitting a PR!