sd-webui-controlnet

sd-webui-controlnet copied to clipboard

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

Trying to use inpaint controlnet and an error appear before generation

Steps to reproduce the problem

1. go to inpaint
2. use inpaint controlnet
3. generate image

What should have happened?

Generate without error

Commit where the problem happens

webui: A1111 / Lshggytiger fork , Zluda controlnet: 1.1.443

What browsers do you use to access the UI ?

No response

Command Line Arguments

--use-zluda --disable-nan-check

List of enabled extensions

animatediff, prompt travel, ADetailler

Console logs

*** Error running process: C:\Zluda-A1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\modules\scripts.py", line 784, in process
        script.process(p, *script_args)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1245, in process
        self.controlnet_hack(p)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1230, in controlnet_hack
        self.controlnet_main_entry(p)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\controlnet.py", line 995, in controlnet_main_entry
        controls, hr_controls = list(zip(*[preprocess_input_image(img) for img in optional_tqdm(input_images)]))
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\controlnet.py", line 995, in <listcomp>
        controls, hr_controls = list(zip(*[preprocess_input_image(img) for img in optional_tqdm(input_images)]))
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\controlnet.py", line 952, in preprocess_input_image
        detected_map, is_image = self.preprocessor[unit.module](
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\utils.py", line 81, in decorated_func
        return cached_func(*args, **kwargs)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\utils.py", line 65, in cached_func
        return func(*args, **kwargs)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\global_state.py", line 37, in unified_preprocessor
        return preprocessor_modules[preprocessor_name](*args, **kwargs)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\processor.py", line 529, in lama_inpaint
        prd_color = model_lama(img_res)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\lama\__init__.py", line 54, in __call__
        result = self.model(image_feed)[0]
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\lama\saicinpainting\training\modules\ffc.py", line 419, in forward
        return self.model(input)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\container.py", line 217, in forward
        input = module(input)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\lama\saicinpainting\training\modules\ffc.py", line 337, in forward
        x_l, x_g = self.conv1((x_l, x_g))
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\lama\saicinpainting\training\modules\ffc.py", line 304, in forward
        x_l, x_g = self.ffc(x)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\lama\saicinpainting\training\modules\ffc.py", line 275, in forward
        out_xg = self.convl2g(x_l) * l2g_gate + self.convg2g(x_g)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\lama\saicinpainting\training\modules\ffc.py", line 198, in forward
        output = self.fu(x)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\venv\lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Zluda-A1111\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\lama\saicinpainting\training\modules\ffc.py", line 86, in forward
        ffted = torch.fft.rfftn(x, dim=fft_dim, norm=self.fft_norm)
    RuntimeError: cuFFT error: CUFFT_INTERNAL_ERROR

Additional information

No response

Can you add "Issue with ZLUDA" in the title?

Also you can report issue here https://github.com/lshqqytiger/stable-diffusion-webui-amdgpu

can confirm same issue on rx6700xt using zluda

EDIT: carefully re-reading, i am NOT experiencing the issue when using inpaint or inpaint global harmony, however issue does appear when using inpaint lama.