stable.art icon indicating copy to clipboard operation
stable.art copied to clipboard

Add ControlNet tile model handling to img2img and inpaint modes

Open Danamir opened this issue 1 year ago • 1 comments

Use alwayson_scripts in payload to submit the img2img or inpaint image to ControlNet 'tile' model. With added controls for ControlNet weight, guidance start, and guidance end, this allow to use a much greater denoising strength when inpainting, and let the tile model handle the image coherency.

To do :

  • [x] Add radio button to select ControlNet mode
  • [x] Add sliders to control CN weight, guidance start, guidance end
  • [x] Alter API payload
  • [x] ~~Remove unwanted returned image by CN API~~
    • The image can be useful, left as is for now
  • [x] Store settings locally
  • [ ] Allow multiple ControlNet modes selected at once (same settings to simplify)
  • [ ] Load controlnet modules from API on launch
  • [ ] Replace hardcoded tile model name by the correct one from the API
  • [ ] Disable ControlNet radio buttons if not found on API side

Danamir avatar May 17 '23 22:05 Danamir

Added tentative support for other ControlNet modules and models. This allow the feature detection, and use of adapted controlnet model.

Associated modules to models :

  • lineart_coarse => lineart
  • pidinet_scribble => scribble
  • openpose_faceonly & openpose_full => openpose

Danamir avatar May 18 '23 00:05 Danamir