[Bug]: controlnet is enabled but no input image is given

[Entretoize]

Have you read the latest version of the FAQ?

• [X] I have visited the FAQ page right now and my issue is not present there

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

Are you using the latest version of the Deforum extension?

• [X] I have Deforum updated to the lastest version and I still have the issue.

What happened?

Getting that error:

 ControlNet - INFO - Loading model from cache: control_v11p_sd15_canny [d14c016b]
*** Error running process: A:\Stable Diffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "A:\Stable Diffusion\stable-diffusion-webui\modules\scripts.py", line 619, in process
        script.process(p, *script_args)
      File "A:\Stable Diffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 977, in process
        self.controlnet_hack(p)
      File "A:\Stable Diffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 966, in controlnet_hack
        self.controlnet_main_entry(p)
      File "A:\Stable Diffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 696, in controlnet_main_entry
        input_image, image_from_a1111 = Script.choose_input_image(p, unit, idx)
      File "A:\Stable Diffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 608, in choose_input_image
        raise ValueError('controlnet is enabled but no input image is given')
    ValueError: controlnet is enabled but no input image is given

And controlnet has no action

Steps to reproduce the problem

1. Deforum extension tab
2. choose interpolation in keyframes tab
3. enable controlnet choose a preprocessor and associated model
4. set an image directory path or image path in "ControlNet Input Video/ Image Path"
5. Click Generate

What should have happened/how would you fix it?

Generation should use the path set in controlnet tab and generate images accordingly, and no error should happen

Torch version

torch: 2.0.1+cu118

On which platform are you launching the webui with the extension?

PC local Windows 11

Deforum settings

https://linkall.pro/share/20231011103808_settings.txt

Webui core settings

https://linkall.pro/share/config.json

Console logs

venv "A:\Stable Diffusion\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.11 | packaged by Anaconda, Inc. | (main, Apr 20 2023, 18:56:50) [MSC v.1916 64 bit (AMD64)]
Version: v1.6.0  
Commit hash: 5ef669de080814067961f28357256e8fe27544f4
Installing xformers
Installing requirements for CivitAI Browser
Launching Web UI with arguments: --xformers --reinstall-xformers --no-half-vae --medvram --listen --enable-insecure-extension-access
Aria2 RPC started
2023-10-11 10:36:03,543 - ControlNet - INFO - ControlNet v1.1.410
ControlNet preprocessor location: A:\Stable Diffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads
2023-10-11 10:36:03,837 - ControlNet - INFO - ControlNet v1.1.410
Loading weights [15012c538f] from A:\Stable Diffusion\stable-diffusion-webui\models\Stable-diffusion\realisticVisionV51_v51VAE.safetensors
*Deforum ControlNet support: enabled*
Creating model from config: A:\Stable Diffusion\stable-diffusion-webui\configs\v1-inference.yaml
Running on local URL:  http://0.0.0.0:7860
Applying attention optimization: xformers... done.
Model loaded in 6.3s (load weights from disk: 1.5s, create model: 0.7s, apply weights to model: 1.6s, calculate empty prompt: 2.3s).

To create a public link, set `share=True` in `launch()`.
Startup time: 44.8s (prepare environment: 15.8s, import torch: 6.9s, import gradio: 1.9s, setup paths: 1.8s, initialize shared: 0.4s, other imports: 1.3s, setup codeformer: 0.4s, load scripts: 9.8s, create ui: 2.2s, gradio launch: 4.4s).
started run_deforum
times_to_run: 1
Deforum extension for auto1111 webui
Git commit: 040da3d8
Starting job task(bw6trpww5upj7j3)-0...
Saving interpolation animation frames to A:\Stable Diffusion\stable-diffusion-webui\outputs\img2img-images\Deforum_20231011103808
Generating interpolated prompts for all frames
Interpolation frame: 0/10
Seed: 127904567
Prompt: tiny cute bunny, vibrant diffraction, highly detailed, intricate, ultra hd, sharp photo, crepuscular rays, in focus --neg nsfw, nude
╭─────┬───╮
│Steps│CFG│
├─────┼───┤
│ 25  │7.0│
╰─────┴───╯
Neither the base nor the masking frames for ControlNet were found. Using the regular pipeline
2023-10-11 10:38:09,070 - ControlNet - INFO - Loading model: control_v11p_sd15_canny [d14c016b]
2023-10-11 10:38:09,916 - ControlNet - INFO - Loaded state_dict from [A:\Stable Diffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\models\control_v11p_sd15_canny.pth]
2023-10-11 10:38:09,917 - ControlNet - INFO - controlnet_default_config
2023-10-11 10:38:13,639 - ControlNet - INFO - ControlNet model control_v11p_sd15_canny [d14c016b] loaded.
*** Error running process: A:\Stable Diffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "A:\Stable Diffusion\stable-diffusion-webui\modules\scripts.py", line 619, in process
        script.process(p, *script_args)
      File "A:\Stable Diffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 977, in process
        self.controlnet_hack(p)
      File "A:\Stable Diffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 966, in controlnet_hack
        self.controlnet_main_entry(p)
      File "A:\Stable Diffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 696, in controlnet_main_entry
        input_image, image_from_a1111 = Script.choose_input_image(p, unit, idx)
      File "A:\Stable Diffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 608, in choose_input_image
        raise ValueError('controlnet is enabled but no input image is given')
    ValueError: controlnet is enabled but no input image is given

---
100%|██████████████████████████████████████████████████████████████████████████████████| 25/25 [00:04<00:00,  5.12it/s]
Interpolation frame: 1/10  ██████████▊                                                | 25/106 [00:03<00:11,  7.10it/s]
Seed: 127904568
Prompt: ( tiny cute bunny, vibrant diffraction, highly detailed, intricate, ultra hd, sharp photo, crepuscular rays, in focus ):0.9666666666666667 AND  ( anthropomorphic clean cat, surrounded by fractals, epic angle and pose, symmetrical, 3d, depth of field ):0.03333333333333333 --neg  ( nsfw, nude):0.9666666666666667 AND  ( nsfw, nude):0.03333333333333333
Deforum progress:  24%|██████████████▊                                                | 25/106 [00:04<00:14,  5.41it/s]
Got a request to stitch frames to video using FFmpeg.
Frames:
A:\Stable Diffusion\stable-diffusion-webui\outputs\img2img-images\Deforum_20231011103808\20231011103808_%09d.png
To Video:
A:\Stable Diffusion\stable-diffusion-webui\outputs\img2img-images\Deforum_20231011103808\20231011103808.mp4
Stitching *video*...
Video stitching done in 0.21 seconds!

Additional information

No response

[moostrik]

I ran into the same problem. The render_interpolation function does not copy the controlnet images to the output folder, hence they are not found when needed.

I did a quick fix by copying some lines from the render_animation function to the render_interpolation fuction:

in render_interpolation, after save_settings_from_animation_run(args, anim_args, parseq_args, loop_args, controlnet_args, video_args, root)

i copied

    if is_controlnet_enabled(controlnet_args):
        unpack_controlnet_vids(args, anim_args, controlnet_args)

and included to the beginning of the file from .deforum_controlnet import unpack_controlnet_vids, is_controlnet_enabled