dream-textures
dream-textures copied to clipboard
carson-katri
Suggestion: Enable Dream Textures in the Compositing Pipeline
Stable Diffusion can use a loose Init image along with a prompt to create a fairly compelling final image. This means that animations using Stable Diffusion are possible (and we've seen them done very well).
Dream Textures is integrated into Blender, meaning that it can operate on the Render Result frame buffer. It would be amazing to push this integration to its logical conclusion, meaning users could block out a scene, then hit render and have Dream Textures process the incoming images.
In order for this to work, Dream Textures would likely need to be integrated into Blender's compositing system as a node. I'm not sure what this would entail, but it would give Dream Textures a much larger use case scenario.
If integration into the Compositor is out of the question, having the ability for Dream Textures to parse a list of images (e.g. an external folder) and process them automatically would be very helpful.
Thanks for the consideration!
I have essentially this request already implemented in #229. Instead of a compositor node (which addons can’t really create) it adds a render pass to cycles which puts an extra output socket in the compositor (like the denoising socket), which is the result of running dream textures on a given frame.
I’ve created a couple test animations already and it’s working pretty well for style transfer!
It would be great to implement dream texture exactly as a filter node.
This will allow you to separate the Depth and color maps, allowing you to create animation variants without correcting the 3D scene.
Also, if the filter nodes can be run one after the other, it will allow you to create a cascade processing of multiple Prompts with different settings.
