sdwebuiapi icon indicating copy to clipboard operation
sdwebuiapi copied to clipboard

How to set up original image and mask for generation in controlnet's inpainting?

Open panghea opened this issue 1 year ago • 1 comments

The following is the code I'm using, where "a.png" is the original image and "b.png" is the image where I've applied a black mask over a white filled area. However, the generated result only outputs the masked image.


from PIL import Image
import webuiapi
api = webuiapi.WebUIApi()
PROMPT="""
masterpiece, best quality, 1girl, face, simple white background
""".strip()
NEGATIVE_PROMPT="""
easynegative
""".strip()
controlnet_units = []
controlnet_image0 = Image.open("a.png") # PIL Image
controlnet_image0mask = Image.open("b.png") # PIL Image

controlnet_unit0 = webuiapi.ControlNetUnit(
    input_image = controlnet_image0,
    module = "inpain_only",
    mask=controlnet_image0mask,
    pixel_perfect=True,
    model = "control_v11p_sd15_inpaint_fp16 [be8bc0ed]",
)
controlnet_units.append(controlnet_unit0)
result1 = api.txt2img(
        prompt=PROMPT, negative_prompt=NEGATIVE_PROMPT, controlnet_units=controlnet_units,
            width=512,
            height=832,)
result1.image.save("test3.png")
result1.images[0].save("test3_1.png")
result1.images[1].save("test3_2.png")

panghea avatar Jun 11 '23 12:06 panghea

Same question. Not sure if this one r.image is the mask image or not? r = api.controlnet_detect(images=[img], module='canny') r.image

derekcbr avatar Nov 04 '23 14:11 derekcbr