[Feature Request]: Transitioning attention

[Glen-O]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

At the moment, you can alter the focus of a particular word using, for example, [(Mario:1.5):(Mario:0.9):0.5], to make it switch from one to the other at the half-way mark. However, sometimes you want a more natural gradient of attention, and it's not really feasible to list out the focus at every step.

I am proposing a transition notation that linearly changes the focus strength.

Proposed workflow

I'm thinking the notation could be (Mario:1.5>0.9) - this indicates that Mario should begin with an attention value of 1.5, and linearly change with each step until it reaches 0.9 at the final step. Note that it would work for increasing as well - (Mario:0.8>1.4). The format would be (Word:StartVal>EndVal).

Additional information

No response

[spinagon]

There's an extension that does something similar, but across several images, not during generation of one. https://github.com/yownas/shift-attention It uses (word:start~end) syntax

[ljleb]

We made an extension that does exactly this: https://github.com/ljleb/prompt-fusion-extension

the syntax is (prompt:1.0,2.0). it takes into account whether it is inside a prompt editing expression and makes sure to span from 1.0 to 2.0 no matter the context (for ex, [(a:1.0,2.0)::5] will transpile to [(a:1.0)::0] [[(a:1.2):0]::1] [[(a:1.4):1]::2] ... (so spans 6 sampler iterations) whether you set the sample steps to 5 or 40. See the wiki for more examples.

The puropose of the extension is to allow to interpolate different types of elements in a prompt. For example, you can interpolate linearly between different prompts, which means interpolating continuously between embeddings in embed space. See the wiki page for the syntax if interested.

Other nice features are variable substitutions. I'll add an entry in the wiki when substitutions are safe to use.

[catboxanon]

Closing as this has been made doable via the extension mentioned above. https://github.com/ljleb/prompt-fusion-extension