Get image preview throught the API

[julien-blanchon]

Hey how can I get the result of latent_preview using the API ?

[julien-blanchon]

Look like from the Python side they get sent to:

https://github.com/comfyanonymous/ComfyUI/blob/a38b9b3ac152fb5679dad03813a93c09e0a4d15e/main.py#L151C1-L159C51

[pzzmyc]

Have you found a way to display the preview image?

[ltdrdata]

Have you found a way to display the preview image?

Your purpose is important. Are you trying to obtain real-time images of the sampling process while sampling is occurring in the KSampler? If that's the case, you must receive them through a websocket.

https://github.com/comfyanonymous/ComfyUI/blob/master/script_examples/websockets_api_example.py

[menguzat]

I'm trying to achieve this, but I couldn't understand WHERE exactly in the example we're getting the preview images?

I'm using nodejs with WS to connect to comfy api, and all I get are JSON objects with the current step, queue, etc.

Should I do something else to expose the previews?

[RandomGitUser321]

I'm trying to figure out something similar as well for a gradio app frontend. Within the websocket portion of my code:

def get_images(ws, prompt):
    prompt_id = queue_prompt(prompt)["prompt_id"]
    output_images = {}
    while True:
        out = ws.recv()
        if isinstance(out, str):
            message = json.loads(out)
            if message["type"] == "executing":
                data = message["data"]
                if data["node"] is None and data["prompt_id"] == prompt_id:
                    break  # Execution is done
            if message["type"] == "progress":
                print("HERE!!!!")
        else:
            continue  # previews are binary data

If you add the:

if message["type"] == "progress":
     print("HERE!!!!")

portion, you'll see it print every step of the generation.

Within the def hijack_progress(server): of the main.py code, you'll see:

server.send_sync("progress", progress, server.client_id)
if preview_image is not None:
     server.send_sync(BinaryEventTypes.UNENCODED_PREVIEW_IMAGE, preview_image, server.client_id)

Using if message["type"] == "progress": works, but I haven't had luck getting if message["type"] == BinaryEventTypes.UNENCODED_PREVIEW_IMAGE: to work, which leads me to believe that the API doesn't generate previews and therefore, that if preview_image is not None: isn't true, so the send_sync never fires off and that means there's likely no preview data to be had.

I even tried adding:

class BinaryEventTypes:
    PREVIEW_IMAGE = 1
    UNENCODED_PREVIEW_IMAGE = 2

to my app and used BinaryEventTypes.UNENCODED_PREVIEW_IMAGE, instead of putting it in quotes, which I mistakenly did before. I also changed the main.py code for comfy to always send that sync and it's still not firing off.

So it definitely seems like comfy doesn't generate previews for the API, which makes sense. However, there should be a way to at least manually trigger it to do so, without having to resort to making some custom node for it.

[n0valis]

Anyone of you got that going? I'm still interested to to it.

[cdrage]

Anyone of you got that going? I'm still interested to to it.

Super late reply, but I tested for dayyyysss trying to figure it out, and WHY I wasn't giving any binary data.

cc @n0valis @RandomGitUser321

Solution:

• Make sure that comfy ui manager is NOT installed. This for some reason "overrides" any previews and prevents any binary data from being sent.
• Set --preview-method explicitly, I did python3 main.py --preview-method taesd --listen 0.0.0.0 --port 3021

After that, I was able to get the binary data following the standard examples.

For example, I used:

def get_images(ws, prompt):
    prompt_id = queue_prompt(prompt)['prompt_id']
    output_images = {}
    current_node = ""
    while True:
        out = ws.recv()
        if isinstance(out, str):
            message = json.loads(out)
            # Print out what instance we got
            print(message)
            if message['type'] == 'executing':
                data = message['data']
                if data['prompt_id'] == prompt_id:
                    if data['node'] is None:
                        break  # Execution is done
                    else:
                        current_node = data['node']
        else:
            print("Got some binary data!")

            # If it's the websocket node, save the image
            if current_node == 'save_image_websocket_node':
                images_output = output_images.get(current_node, [])
                images_output.append(out[8:])
                output_images[current_node] = images_output
            
            # We save everything after the first 8 bytes, which is the message type
            bytesIO = io.BytesIO(out[8:])
            image = Image.open(bytesIO)
            
            # Let's save the preview image to /tmp/preview.png
            image.save("/tmp/comfy_preview.png", format="PNG")
            print("Saved the preview image to /tmp/comfy_preview.png")