How do I make an API call to the controlnet img2img without using preprocessors?

[SwayStar123]

Im trying to write a program to call the controlnet img2img api using the rest endpoint. However the results seem to be differing from the UI even using the same seed

These are the values im passing to the endpoint

let payload = json!({
        "prompt": "1girl, brown hair, red dress, red ribbon, white sleeves",
        "steps": 20,
        "init_images": [base64_mmd],
        // "controlnet_module": "depth",
        "controlnet_model": "control_sd15_depth",
        "controlnet_input_image": [base64_og],
        // "controlnet_mask": [base64_og],
        "sampler_index": "DPM++ SDE Karras",
        "restore_faces": false,
        "width": 480,
        "height": 360,
        "controlnet_guessmode": false,
        "denoising_strength": 0.75,
        "seed": 3570635895_u32,
    });

As far as I understand, "controlnet_module" is the name of the preprocessor. However as I am doing the preprocessing myself I do not need this so I have commented it out. Someone said that the "controlnet_mask" might be what I need to use if I want to input an image without preprocessing, however that doesnt seem to be the case either as a "controlnet_input_image" value seems to be mandatory. Inputing just the preprocessed image doesnt seem to work either.

What am I missing?

For reference, with the same seed, and for the following depth map the UI generates: However my program generates:

Supplying both the mask and input image (as the same image) doesnt seem to work either, both result in nearly identical outputs

[SwayStar123]

Okay I think it mightve been the "Allow other script to control this extention" setting being turned off. I turned it on, however getting a weirder error

using mask as input
Loading preprocessor:
Error running process: D:\Programming\StableDiffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py
Traceback (most recent call last):
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\modules\scripts.py", line 386, in process
    script.process(p, *script_args)
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 609, in process
    preprocessor = self.preprocessor[module]
KeyError: ''

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\anyio\streams\memory.py", line 94, in receive
    return self.receive_nowait()
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\anyio\streams\memory.py", line 89, in receive_nowait
    raise WouldBlock
anyio.WouldBlock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 77, in call_next
    message = await recv_stream.receive()
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\anyio\streams\memory.py", line 114, in receive
    raise EndOfStream
anyio.EndOfStream

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 407, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\fastapi\applications.py", line 271, in __call__
    await super().__call__(scope, receive, send)
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\starlette\applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 184, in __call__
    raise exc
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 106, in __call__
    response = await self.dispatch_func(request, call_next)
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\modules\api\api.py", line 96, in log_and_time
    res: Response = await call_next(req)
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 80, in call_next
    raise app_exc
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 69, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\gzip.py", line 24, in __call__
    await responder(scope, receive, send)
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\gzip.py", line 44, in __call__
    await self.app(scope, receive, self.send_with_gzip)
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
    raise exc
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in __call__
    raise e
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 706, in __call__
    await route.handle(scope, receive, send)
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\starlette\routing.py", line 66, in app
    response = await func(request)
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\fastapi\routing.py", line 237, in app
    raw_response = await run_endpoint_function(
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\venv\lib\site-packages\fastapi\routing.py", line 163, in run_endpoint_function
    return await dependant.call(**values)
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\api.py", line 322, in img2img
    processed = process_images(p)
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\modules\processing.py", line 486, in process_images
    res = process_images_inner(p)
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\modules\processing.py", line 713, in process_images_inner
    extra_networks.deactivate(p, extra_network_data)
UnboundLocalError: local variable 'extra_network_data' referenced before assignment

This error is occuring even with the UI and not just in my program

[Mikubill]

Use "none" preprocessor if you want to pipe the input image to ControlNet as is.

[SwayStar123]

Use "none" preprocessor if you want to pipe the input image to ControlNet as is.

"controlnet_module" is the preprocessor right? Tried None for it just now, same error as above reply

[SwayStar123]

Okay restarting it seems to have made it work. It still does not seem to be following the depth image.

Here is the image for reference

and here is my updated payload

let payload = json!({
        "prompt": "1girl, brown hair, red dress, red ribbon, white sleeves",
        "negative_prompt": "ugly",
        "steps": 20,
        "init_images": [base64_mmd],
        "controlnet_module": "None",
        "controlnet_model": "control_sd15_depth [fef5e48e]",
        "controlnet_input_image": [base64_og],
        "controlnet_guidance": 1.0,
        "sampler_index": "DPM++ SDE Karras",
        "restore_faces": false,
        "width": 480,
        "height": 360,
        "controlnet_guessmode": false,
        "denoising_strength": 0.95,
        "cfg_scale": 7,
    });

base64_og is the depth map image, and the base64_mmd is the input image to the img2img

Also im getting this error, which I do not get whilst using the UI (with none as the preprocessor)

Loading preprocessor: None
Error running process: D:\Programming\StableDiffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py
Traceback (most recent call last):
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\modules\scripts.py", line 386, in process
    script.process(p, *script_args)
  File "D:\Programming\StableDiffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 609, in process
    preprocessor = self.preprocessor[module]
KeyError: 'None'

100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 11.35it/s]

[kft334]

It's case sensitive I think. Try using 'none' instead of 'None'

[SwayStar123]

It's case sensitive I think. Try using 'none' instead of 'None'

Cant believe im so dumb