stable-diffusion-webui icon indicating copy to clipboard operation
stable-diffusion-webui copied to clipboard

Add patchmatch

Open catboxanon opened this issue 2 years ago • 0 comments

Describe what this pull request is trying to achieve.

This implements patchmatch which should allow for much better inpainting (and more notably outpainting) results.

Closes https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/4681

Additional notes and description of your changes

Requires pypatchmatch to be installed. This is currently only available on Windows via pip. Further install instructions are here.

Mask blur must be set to 0, because blurring breaks the seams that patchmatch tries to recover. There's probably better ways to blend the final image but I'm not sure what the correct solution for this should be. If you want you can play around with the individual passes before the final composite with https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/8814.

I've also hardcoded the patch_size parameter to 4. Might want to expose this to the user, but increasing it also greatly increases computation time (the default is 15 which takes far too long to run for practical purposes).

Haven't tested with the openOutpaint extension yet but I'm hoping this addition could make big gains for that extension. Needs 4: "patch match" added after this line. I think the extension does not currently expose the denoise parameter either, which probably needs to be done to yield good results.

Opening as a PR for now since the installation process should be smoothed out before merging, and to see if it even works well in practice. Feedback appreciated.

Environment this was tested in

  • OS: Windows
  • Browser: Firefox
  • Graphics card: NVIDIA RTX 3090

catboxanon avatar Mar 24 '23 01:03 catboxanon