stable-diffusion-webui icon indicating copy to clipboard operation
stable-diffusion-webui copied to clipboard

Animation generating with ~ tilde prompt in txt2img

Open machineinthegod opened this issue 3 years ago • 5 comments

Implemented support for creating animation between two prompts using tilde ~ in txt2img. Most animation logic is in animate.py, there's important addition to KDiffusionSampler class (animation only works with those). Rest of the changes are additional tab for viewing the video in txt2img in gradio, and small changes in process_images and txt2img to pass the parameters.

(It's not naïve animation generation, like img2img loopback methods, there's logic to try and get as smooth animation as possible instead of jumpy animation with noise and things randomly popping in and out of existence between frames)

Requires getting openh264 codec in windows to encode the mp4 video (but the frames are generated either way).

machineinthegod avatar Sep 11 '22 20:09 machineinthegod

Hello @machineinthegod thanks for your contribution!

New PRs are opened against dev branch.

codedealer avatar Sep 11 '22 20:09 codedealer

@machineinthegod, do you mind sharing example result? Would like to learn more about this. Thanks in advance both for the PR and response.

hafiidz avatar Sep 12 '22 08:09 hafiidz

how this whould work? exaplem of a prompt please

dudumaroja avatar Sep 14 '22 02:09 dudumaroja

Hello @machineinthegod thanks for your contribution!

New PRs are opened against dev branch.

Conflicts changing it to dev, leaving it for now.

@machineinthegod (reproducible) examples would be helpful

hlky avatar Sep 14 '22 06:09 hlky

This works well, but I suggest a few things before merge. first, the process images fails at times. the animation process should save off each image as it is generated, rather than store then all into ram. thoughts on getting this feature merged so I can start to implement more changes?

ghost avatar Sep 16 '22 13:09 ghost