prompt-morph icon indicating copy to clipboard operation
prompt-morph copied to clipboard

Add alternate morphing functions to prompt-morph.

Open kdreibel opened this issue 2 years ago • 0 comments

Overview

prompt_morph.py currently always morphs between each prompt in a linear way. This PR adds some alternate morphing functions in order to spend more/less time at keyframes, depending on the user's preference.

Comparison

Here is a series of gifs illustrating the difference. The gifs are stuck together so that the keyframes should always synchronize. The circle and square are the keyframes, the hands (?) generally appear in the middle.

A (red circle|blue square), viewed from the side. Clip art. Steps: 28, Sampler: Euler a, CFG scale: 28.5, Seed: 713180077

morphin-time

From left-to-right: Linear, Sine, S-Parabola, Parabolic, Parabolic Bounce

x/y graphs

Linear

This is the current behavior.

linear

Sine (but secretly Cosine)

Start slow, speed through the middle, then slow again. The difference from Linear is barely noticeable, though.

sine

S-parabola

Start and end fast, linger in the middle for a bit. NOTE: I am pretty sure there is a proper mathematical name for this, but I don't know it.

S-Parabola

Parabolic

Start slow and accelerate. Drop back to zero velocity on every keyframe.

parabolic

Parabola bounce

Alternate between accelerating and decelerating. If linear would cause a ball to move up-down-up at steady velocity, Parabola bounce would cause it to look like it was bouncing off the ground like a normal ball affected by gravity. In the gifs above, you can see that (4) and (5) are identical from circle to square, but then differ on the subsequent morph.

Parabolic bounce

Discussion / Rationale

  • My theory was that if there was a morph where e.g. a basketball went from someone's hand to the ground, the parabolic bounce would allow it to better simulate gravity. But then it turned out I couldn't find any sort of prompt pair to generate that kind of behavior.
  • Another use case was for animating body movement. person with arms up -> person with arms down. The morphs didn't produce expected movement though.
  • I guess my question is, the original rationale behind this PR didn't pan out as expected, but is the functionality still useful?

Future work

  • I liked the idea of the Sine function spending more time near the keyframes, and "glossing over" the chaotic parts between keyframes. But it's hardly noticeable. Perhaps I'll revisit this to find a better function to meet that criteria.
  • On the other hand, maybe we should scrap the dropdown entirely, and support the complex inline math that Deforum uses?

Also also wik

  • A line starting with # will be treated as a comment, and ignored.

kdreibel avatar Nov 13 '22 03:11 kdreibel