[enhancement]: color prompt

[laurentopia]

Is there an existing issue for this?

• [X] I have searched the existing issues

Contact Details

No response

What should this feature add?

input: a rough sketch where each zone has a flat color prompt workflow: sample each flat color create a prompt text input, enter prompt for each color output: an image that follows the prompt per color region

Alternatives

rolling dice a thousand times and hope you get lucky on masked canvas

Aditional Content

No response

[hipsterusername]

Relevant research paper, for when we implement this

https://github.com/omerbt/MultiDiffusion

[laurentopia]

Something no model does well is directed flow of liquid. As an example draw a thick wavy mask and ask for a flow of water. The NN are not trained on understand flowing liquids perhaps. Maybe it's not possible with SD to teach it flow but I've seen NN guided by image gradient so maybe an extra layer to explain direction would help. Similar to flow map to describe liquids flowing in a shader.