InvokeAI icon indicating copy to clipboard operation
InvokeAI copied to clipboard

Add a `TiledStableDiffusionRefineInvocation` (for upscaling workflows)

Open RyanJDick opened this issue 1 year ago • 0 comments

Summary

Adds a TiledStableDiffusionRefineInvocation node with the following characteristics:

  • The node applies the following sequence:
    • Split tiles in image space
    • VAE encode
    • Denoise
    • VAE decode
    • Merge tiles in image space
  • The same process can be achieved in a workflow with existing nodes. The primary reason for creating this standalone node is to avoid the need for an 'iterate' node to achieve this workflow. Iterate nodes have some limitations, and are not currently permitted in the hosted InvokeAI app.
  • The TiledMultiDiffusionDenoiseLatents node serves a similar purpose in upscaling workflows. Here are the rough tradeoffs between them:
    • MultiDiffusion upscaling requires using a tiled VAE, which can cause some discoloration issues
    • MultiDiffusion is less prone to visible seams.
    • TODO: Speed comparison

TODO: Add comparison images.

Related Issues / Discussions

QA Instructions

Merge Plan

Checklist

  • [ ] The PR has a short but descriptive title, suitable for a changelog
  • [ ] Tests added / updated (if applicable)
  • [ ] Documentation added / updated (if applicable)

RyanJDick avatar Jul 15 '24 21:07 RyanJDick