open-webui
feat: ComfyUI Custom Workflow Support
Is your feature request related to a problem? Please describe.
The feature will enhance the text to image capability of OpenWebUI. Especially for company using Multimedia and Image/video content creation pipeline. By enabling this feature the user could use multiple workflow not only to generate image from text, but would open multiple other image generation custom features. Compatible ComfyUI Workflow could be shared within the OpenWebUI Prompt/ModelFile webpage.
Describe the solution you'd like
The option to load an Exported workflow into the setting windows of OpenWebUI. With a few basic setting exposed: Dimension, steps, models, etc... (like it is already the case with the current integration)
Describe alternatives you've considered
Manually update the comfyui.py file in the repo for a hardcoded alternative
I did try to manually update the comfyui.py file with a basic SDXL + refiner workflow that works well, but it does not seems to work. I get the following error when I click the generate image button:
FYI - I manage to make a custom ComfyUI work flow work! I was missing a "," and also as ComfyUI nodes ID are generated randomly depending at which moment you create the node, it needs to be updated accordingly in the code to point the variable for the prompt, model, steps and dimension.
Below is the updated comfyui.py file that can be replaced in /app/backend/apps/images/utils/comfyui.py. comfyui.py.zip
If you have enough ressource to load SDXL + the REFINER on a machine it provide pretty high quality image as output. Just make sure you download the 2 models below in your ComfyUI checkpoint server before inferring. Setup the settings like the screenshot attached for best results.
@Michelklingler I second this request.
I have different workflows for different kinds of output types and subjects. Would be great to switch between them as needed so that I can get high quality results for each rather than subpar one-size-fits-all as it is right now. Love having comfyui as a part of the application, so this would just make it that much better.
EDIT: I looked into this some more and found that this is offered as part of an add-on for ComfyUI that provides API endpoints that list workflows: https://github.com/pythongosssss/ComfyUI-Custom-Scripts
It does not appear to yet be something that ComfyUI can provide out of the box.
This ComfyUI add-on uses a couple of api endpoints that could facilitate: Provide a list of workflows by name: /pysssss/workflows Get the defintion of a workflow: pysssss/workflows/workflownameFromPreviousEndpoint
The output from the second endpoint could be manipulated and inserted into the request script in place of the default workflow.
The relevant add-on files are here: https://github.com/pythongosssss/ComfyUI-Custom-Scripts/blob/main/web/js/workflows.js#L50
I don't know if I'll have an opportunity to write anything out to support this but it's definitely of interest to me.
it doesn't work this way. The change to support this is more complex. Each workflow is unique. So either the open-webui team sets up some standards what need to be in a workflow or we put a mapping mechanism in place that let's a user use any workflow he likes put it in some folder in open-webui and a configuration file "xpath" or some other mechanism to tell open-webui where to put the negative prompt ... and so on. So many options on how to advance this. If you even be possible to give the LLM tools/functions it can call, to customize the negative prompt, model, seed and so on, from within the chat
It may be possible to support some special workflows in the form of lists, such as SDXL, LCM, cascade, etc.
