stable-diffusion-webui
stable-diffusion-webui copied to clipboard
Launch api on separate fastapi app from webui
This moves the API to a separate uvicorn server when launched alongside the webui frontend. The API and webui servers still share the same backend objects (so no additional VRAM is used to load duplicative models). This change brings back the API docs and more importantly avoids silent, unexpected failure of API calls,
Adresses https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/10136 The latest gradio update to 3.28.1 removed API docs and in some cases causes silent failure of API calls. As reported by OP in issue #10136: "After more investigating seems like the API is broken as well. My testing is limited, but it seems that the API requests are still being received and jobs are being ran, however the responses back never happen and it times out." This is supported by what Abidlabs (gradio author) said in https://github.com/gradio-app/gradio/issues/4054 "So long story short, we don't encourage the use of the HTTP API endpoints directly. They have several issues, such as timeouts for long predictions and the fact that they don't work well with the queue if queuing is enabled."
Gradio has a special way of launching that may cause the mounted API app to break in silent, unexpected ways. I believe the sustainable solution here is to simply launch the API in its own uvicorn server. If the user needs the API server and webui frontend served on the same port, they could use a NGinx reverse proxy.
Running the API uvicorn server in a separate thread requires special consideration discussed here https://stackoverflow.com/questions/61577643/python-how-to-use-fastapi-and-uvicorn-run-without-blocking-the-thread In my PR, I used gradio's solution from gradio.networking.Server
The current commit is just a proof of concept to see if this is a change you would consider making. If its helpful, I wanted to make a more aggressive refactoring of the api_only function and move the uvicorn server threading into the api class itself, but these touch alot more lines of code.
+1 want to use the api and can't
Running on a different port would be a degradation for projects that integrate the webui into something - are you sure this fixes something -- ie problems do exist with the way it currently works and they definitely go away with the switch to a separate server?
Is there a way to reproduce the problem you're describing reliably?
Hi good question, I don't know how to reproduce the issue so I'll close the pr.