cog icon indicating copy to clipboard operation
cog copied to clipboard

Run predictions off main thread to avoid blocking health check

Open ggilder opened this issue 8 months ago • 1 comments

Fixes https://github.com/replicate/cog/issues/1719

Defining the prediction endpoints with async def runs them on the main thread per FastAPI docs, which is problematic because it blocks the server from responding to the health check endpoint. Converting these to def allows health checks to run and fixes the problem I described in the above issue.

ggilder avatar Jun 06 '24 21:06 ggilder