Dynamic prompt updates in screen example

[jcelerier]

I'm trying to make the prompt in the screen example evolve over time. However, if I move

        stream.prepare(
            prompt=prompt,
            negative_prompt=negative_prompt,
            num_inference_steps=50,
            guidance_scale=guidance_scale,
            delta=delta,
        )

from before the main loop to within it, just before the call to

        output_images = stream.stream(
            input_batch.to(device=stream.device, dtype=stream.dtype)
        ).cpu()

then my output becomes pretty much some random uniform color.

If I add a timer to call stream.prepare for instance once every second, then once every second I get some grey / blank frame in the middle of the stream. How can I make this more fluid and continuous ?

[olegchomp]

There is function for updating prompt only without preparing the whole stream

https://github.com/cumulo-autumn/StreamDiffusion/blob/765d71029b1404b94aee2865178d71c257c20318/src/streamdiffusion/pipeline.py#L255-L262

[jcelerier]

hmm does not seem to work very well here if I replace my call to prepare:

[E] 3: [executionContext.cpp::validateInputBindings::2046] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::validateInputBindings::2046, condition: profileMinDims.d[i] <= dimensions.d[i]. Supplied binding dimension [2,77,1024] for bindings[2] exceed min ~ max range at index 0, maximum dimension in profile is 4, minimum dimension in profile is 4, but supplied dimension is 2.

this gives me a lot of visual artifacts too

[olegchomp]

i mean, you need to prepare stream anyway, but after you can change prompt with function above.

[jcelerier]

I do, I first have a call to prepare() unchanged from screen.py, and then I call stream.update_prompt("blah") whenever I get a new prompt