[Bug]: error when i use the addetailer and the controlnet extension at the same time and with a batch count more than 1

[God-ALK]

Checklist

• [ ] The issue exists after disabling all extensions
• [X] The issue exists on a clean installation of webui
• [X] The issue is caused by an extension, but I believe it is caused by a bug in the webui
• [X] The issue exists in the current version of the webui
• [ ] The issue has not been reported before recently
• [ ] The issue has been reported before but has not been fixed yet

What happened?

cannot use addetailer and controlnet at the same time

Steps to reproduce the problem

1 Enable the addetailer and the controlnet extension with a batch count more than 1 2 Generate the image, the first image it's generated ok but after that the error appears

What should have happened?

I used the webui the last year and this never happen to me, sorry for the bad english

What browsers do you use to access the UI ?

Mozilla Firefox

Sysinfo

sysinfo-2024-05-14-16-23.json

Console logs

*** Error completing request
*** Arguments: ('task(5hyjlx50s7joyev)', <gradio.routes.Request object at 0x000001D27E826F50>, '1girl, nude, navel, (flat chest), groin, masterpiece, highres, white hair, short hair, parted bangs, lips, red eyes, looking at viewer, simple background, black background, dark background, no sunlight, soft light, sweat, 4k', 'ugly, low quality, lowres', [], 2, 1, 5, 768, 512, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', 'Automatic', '', '', [], 0, 25, 'DPM++ SDE', 'Automatic', False, '', 0.8, -1, False, -1, 0, 0, 0, True, False, {'ad_model': 'face_yolov8n.pt', 'ad_model_classes': '', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.5, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M', 'ad_scheduler': 'Use same scheduler', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': True, 'ad_clip_skip': 2, 'ad_restore_face': False, 'ad_controlnet_model': 'control_v11p_sd15_inpaint_fp16 [be8bc0ed]', 'ad_controlnet_module': 'inpaint_global_harmonious', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'None', 'ad_model_classes': '', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M', 'ad_scheduler': 'Use same scheduler', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, ControlNetUnit(is_ui=True, input_mode=<InputMode.SIMPLE: 'simple'>, batch_images='', output_dir='', loopback=False, enabled=False, module='scribble_pidinet', model='control_lora_rank128_v11p_sd15_scribble_fp16 [131fe54f]', weight=0.6, image={'image': array([[[ 3,  3,  4],
***         [ 3,  1,  5],
***         [ 1,  1,  3],
***         ...,
***         [ 7, 10, 12],
***         [ 5,  9, 12],
***         [ 7, 12, 14]],
***
***        [[ 2,  1,  4],
***         [ 1,  1,  1],
***         [ 1,  2,  2],
***         ...,
***         [ 7, 10, 15],
***         [ 5, 10, 13],
***         [ 6, 11, 13]],
***
***        [[ 2,  1,  4],
***         [ 3,  2,  3],
***         [ 2,  2,  2],
***         ...,
***         [ 7, 10, 15],
***         [ 7, 11, 15],
***         [ 5,  9, 15]],
***
***        ...,
***
***        [[ 2,  2,  4],
***         [ 2,  1,  3],
***         [ 2,  1,  2],
***         ...,
***         [ 4,  4,  8],
***         [ 4,  5,  7],
***         [ 4,  5,  8]],
***
***        [[ 2,  3,  2],
***         [ 0,  1,  0],
***         [ 0,  2,  1],
***         ...,
***         [ 4,  5,  7],
***         [ 3,  4,  7],
***         [ 5,  5,  8]],
***
***        [[ 0,  3,  3],
***         [ 0,  0,  0],
***         [ 1,  0,  2],
***         ...,
***         [ 2,  3,  6],
***         [ 3,  3,  7],
***         [ 2,  5,  7]]], dtype=uint8), 'mask': array([[[0, 0, 0],
***         [0, 0, 0],
***         [0, 0, 0],
***         ...,
***         [0, 0, 0],
***         [0, 0, 0],
***         [0, 0, 0]],
***
***        [[0, 0, 0],
***         [0, 0, 0],
***         [0, 0, 0],
***         ...,
***         [0, 0, 0],
***         [0, 0, 0],
***         [0, 0, 0]],
***
***        [[0, 0, 0],
***         [0, 0, 0],
***         [0, 0, 0],
***         ...,
***         [0, 0, 0],
***         [0, 0, 0],
***         [0, 0, 0]],
***
***        ...,
***
***        [[0, 0, 0],
***         [0, 0, 0],
***         [0, 0, 0],
***         ...,
***         [0, 0, 0],
***         [0, 0, 0],
***         [0, 0, 0]],
***
***        [[0, 0, 0],
***         [0, 0, 0],
***         [0, 0, 0],
***         ...,
***         [0, 0, 0],
***         [0, 0, 0],
***         [0, 0, 0]],
***
***        [[0, 0, 0],
***         [0, 0, 0],
***         [0, 0, 0],
***         ...,
***         [0, 0, 0],
***         [0, 0, 0],
***         [0, 0, 0]]], dtype=uint8)}, resize_mode=<ResizeMode.INNER_FIT: 'Crop and Resize'>, low_vram=False, processor_res=512, threshold_a=0.5, threshold_b=0.5, guidance_start=0.0, guidance_end=1.0, pixel_perfect=False, control_mode=<ControlMode.BALANCED: 'Balanced'>, inpaint_crop_input_image=False, hr_option=<HiResFixOption.BOTH: 'Both'>, save_detected_map=True, advanced_weighting=None, effective_region_mask=None, pulid_mode=<PuLIDMode.FIDELITY: 'Fidelity'>, ipadapter_input=None, mask=None, batch_mask_dir=None, animatediff_batch=False, batch_modifiers=[], batch_image_files=[]), ControlNetUnit(is_ui=True, input_mode=<InputMode.SIMPLE: 'simple'>, batch_images='', output_dir='', loopback=False, enabled=False, module='none', model='None', weight=1.0, image=None, resize_mode=<ResizeMode.INNER_FIT: 'Crop and Resize'>, low_vram=False, processor_res=-1, threshold_a=-1.0, threshold_b=-1.0, guidance_start=0.0, guidance_end=1.0, pixel_perfect=False, control_mode=<ControlMode.BALANCED: 'Balanced'>, inpaint_crop_input_image=False, hr_option=<HiResFixOption.BOTH: 'Both'>, save_detected_map=True, advanced_weighting=None, effective_region_mask=None, pulid_mode=<PuLIDMode.FIDELITY: 'Fidelity'>, ipadapter_input=None, mask=None, batch_mask_dir=None, animatediff_batch=False, batch_modifiers=[], batch_image_files=[]), ControlNetUnit(is_ui=True, input_mode=<InputMode.SIMPLE: 'simple'>, batch_images='', output_dir='', loopback=False, enabled=False, module='none', model='None', weight=1.0, image=None, resize_mode=<ResizeMode.INNER_FIT: 'Crop and Resize'>, low_vram=False, processor_res=-1, threshold_a=-1.0, threshold_b=-1.0, guidance_start=0.0, guidance_end=1.0, pixel_perfect=False, control_mode=<ControlMode.BALANCED: 'Balanced'>, inpaint_crop_input_image=False, hr_option=<HiResFixOption.BOTH: 'Both'>, save_detected_map=True, advanced_weighting=None, effective_region_mask=None, pulid_mode=<PuLIDMode.FIDELITY: 'Fidelity'>, ipadapter_input=None, mask=None, batch_mask_dir=None, animatediff_batch=False, batch_modifiers=[], batch_image_files=[]), False, False, 'positive', 'comma', 0, False, False, 'start', '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, False, False, False, 0, False, None, None, False, None, None, False, None, None, False, 50) {}
    Traceback (most recent call last):
      File "C:\Users\acer\sd.webui\webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "C:\Users\acer\sd.webui\webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "C:\Users\acer\sd.webui\webui\modules\txt2img.py", line 109, in txt2img
        processed = processing.process_images(p)
      File "C:\Users\acer\sd.webui\webui\modules\processing.py", line 845, in process_images
        res = process_images_inner(p)
      File "C:\Users\acer\sd.webui\webui\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 59, in processing_process_images_hijack
        return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
      File "C:\Users\acer\sd.webui\webui\modules\processing.py", line 981, in process_images_inner
        samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
      File "C:\Users\acer\sd.webui\webui\extensions\sd-webui-controlnet\scripts\hook.py", line 463, in process_sample
        return process.sample_before_CN_hack(*args, **kwargs)
      File "C:\Users\acer\sd.webui\webui\modules\processing.py", line 1328, in sample
        samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
      File "C:\Users\acer\sd.webui\webui\modules\sd_samplers_kdiffusion.py", line 218, in sample
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "C:\Users\acer\sd.webui\webui\modules\sd_samplers_common.py", line 272, in launch_sampling
        return func()
      File "C:\Users\acer\sd.webui\webui\modules\sd_samplers_kdiffusion.py", line 218, in <lambda>
        samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "C:\Users\acer\sd.webui\webui\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "C:\Users\acer\sd.webui\webui\repositories\k-diffusion\k_diffusion\sampling.py", line 553, in sample_dpmpp_sde
        denoised = model(x, sigmas[i] * s_in, **extra_args)
      File "C:\Users\acer\sd.webui\webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "C:\Users\acer\sd.webui\webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\acer\sd.webui\webui\modules\sd_samplers_cfg_denoiser.py", line 269, in forward
        devices.test_for_nans(x_out, "unet")
      File "C:\Users\acer\sd.webui\webui\modules\devices.py", line 255, in test_for_nans
        raise NansException(message)
    modules.devices.NansException: A tensor with all NaNs was produced in Unet. This could be either because there's not enough precision to represent the picture, or because your video card does not support half type. Try setting the "Upcast cross attention layer to float32" option in Settings > Stable Diffusion or using the --no-half commandline argument to fix this. Use --disable-nan-check commandline argument to disable this check.

Additional information

i reinstall the webui just a few days ago

[JeffoDogge]

Same issue here. "Upcast cross attention layer to float32" has no effect and neither does --no-half. Generate forever with batch count == 1 does still work though (as a sort of workaround).

[Gem717]

Same issue as well. --disable-nan-check gives black pics (last 3 out of 4), exactly the same as how God-ALK describes it. the --no-half commandline argument did not help.

Did install kohya-ss a few days ago, before that there was no issue with SD.

[AnotherAsian123]

Also having the same issue. System:

• CPU: 5950x
• GPU: 3090

Stuff I tried:

1. CUDA: Updating to 12.4 had no effect
2. Models: Not isolated to any singular model, happens with all I tried
3. Commands: --no-half, --no-half-vae, error still happens
4. VAE: Trying different VAEs doesn't have any effect
5. Upcast: Doesn't fix, worse performance
6. Render size: Lowering size doesnt work
7. CUDNN: Updating had no effect
8. XFORMERS: Enabling/disabling had no effect
9. Image generation: Switching from XYZ to batch size had no effect
10. Clean install: Deleted everything, fresh reinstall, still same issue.

It seems like Adetailer is the culprit in my case as I can get around multiple image generation by disabling Adetailer, but just confused because before a certain point (not sure which update) I had no issues using Adetailer.

[kodlamaicin]

Hi Friend, I just signed up to help you. It will work fine if you uninstall the version you have installed and install this version instead. Download version v23.9.1 from this link https://gitee.com/jovan_qiao/adetailer/tree/v23.9.1/ ;)

[God-ALK]

Hi Friend, I just signed up to help you. It will work fine if you uninstall the version you have installed and install this version instead. Download version v23.9.1 from this link https://gitee.com/jovan_qiao/adetailer/tree/v23.9.1/ ;)

OMG THANKS BUDDY 🙏🏻

[AnotherAsian123]

Adetailer seems to be working again, whatever recent update that happened. I can now generate multiple images with Adetailer enabled. Did not have install from that gitee link.

[furiousjay]

I have the same problem. I even tried a complete fresh install of Auto1111. I am getting this error in the following scenario:

• contronet batch is used
• adetailer is used
• Multiple "Prompts from File or Textbox" option in Auto1111 is used

After the first Contronet image is done for the first prompt, the process aborts.