Fails when using ADetailer inpainting (50 series)

[mxicat]

pytorch version: 2.8.0.dev20250420+cu128 Device: cuda:0 NVIDIA GeForce RTX 5090 : native

The problem is like this: https://github.com/Bing-su/adetailer/issues/733

Only face_yolov8n.pt & hand_yolov8n.pt could be used properly. When using any other models, will show the error message below and the result will be black images.

F:\stable-diffusion-webui-forge\modules\processing.py:1040: RuntimeWarning: invalid value encountered in cast x_sample = x_sample.astype(np.uint8)

Steps to reproduce:

1. Install ADetailer
2. Use adetailer to inpaint an image using face_yolov8n.pt, the result is good
3. Generate another image with another models
4. "invalid value encountered in cast": The detection area of the image will be black, and any image after will be completely black.

Logs can be seen here https://github.com/Bing-su/adetailer/issues/790

[MisterChief95]

I suggest opening an issue for this over on the adetailer repository. The issue likely arises from some dependency incompatibilities they may have rather than anything happening within Forge. It's important to note that while Pytorch 2.7+ is required for Blackwell GPUs (5000 series), Forge technically still recommends Pytorch 2.3.1 and some versions around that.

[mxicat]

I suggest opening an issue for this over on the adetailer repository. The issue likely arises from some dependency incompatibilities they may have rather than anything happening within Forge. It's important to note that while Pytorch 2.7+ is required for Blackwell GPUs (5000 series), Forge technically still recommends Pytorch 2.3.1 and some versions around that.

I previously opened this issue on the Adetailer repository and received a response indicating that the problem is not related to Adetailer itself but rather to the sampler. When using A1111, the extension functions properly without encountering this issue, which suggests that it may be caused by a feature or behavior specific to Forge. I am posting here simply to point out this issue in the hope that it can be addressed in a future update. I am not very familiar with this platform, so if this post causes any inconvenience, I sincerely apologize. Thank you for your patience.

[MisterChief95]

No need to apologize! Lets see if we can figure it out. I hopped over to the adetailer repo to look at your issue. Its unfortunate that there isn't more error info in your logs, but at the very least we can confirm that its not Adetailer having an issue (adetailer prints out huge formatted error messages, which is not the case here).

Couple Questions:

1. When using Automatic1111, did you have any other extensions enabled?
2. Can you share which sampler/scheduler you're using?
3. Is it safe to assume this issue doesn't happen during normal generation? Only during Adetailer? What about HiRes Fix/Img2Img?

Also, small personal suggestion, I can see you're using a majority of your VRAM for weights. My suggestion is to lower this as much as possible while still keeping the entire checkpoint + LoRAs in VRAM. Since you're using Illustrious a good number would be 15000-ish. This lets the remaining VRAM be used for inference (the actual image generation), which means you can run larger batch sizes + generate much larger images in one go without saturating VRAM and having to offload to system RAM.

[mxicat]

No need to apologize! Lets see if we can figure it out. I hopped over to the adetailer repo to look at your issue. Its unfortunate that there isn't more error info in your logs, but at the very least we can confirm that its not Adetailer having an issue (adetailer prints out huge formatted error messages, which is not the case here).

Couple Questions:

1. When using Automatic1111, did you have any other extensions enabled?
2. Can you share which sampler/scheduler you're using?
3. Is it safe to assume this issue doesn't happen during normal generation? Only during Adetailer? What about HiRes Fix/Img2Img?

Also, small personal suggestion, I can see you're using a majority of your VRAM for weights. My suggestion is to lower this as much as possible while still keeping the entire checkpoint + LoRAs in VRAM. Since you're using Illustrious a good number would be 15000-ish. This lets the remaining VRAM be used for inference (the actual image generation), which means you can run larger batch sizes + generate much larger images in one go without saturating VRAM and having to offload to system RAM.

Thank you for your reply. Regarding your questions:

1. The extensions I use on both WebUI versions are the same (as shown below):

2. Euler a / Normal. I have tried other samplers, including DPM++ and DDIM, but the problem remains the same.

3. I did not encounter this problem when using img2img, img2img inpaint, or HiRes. I only noticed the issue occurring when using ADetailer.

Thank you for your advice. I will try adjusting the settings.

[MisterChief95]

Nothing seems out of the ordinary. Though another small suggestion is that I don't think you need use sd-webui-infolk-prompt-comment anymore. I'm fairly certain A1111/Forge have # comments in prompts built-in now. You can search "comment" in the settings tab to make sure its enabled.

Use adetailer to inpaint an image using face_yolov8n.pt, the result is good

Which adetailer models don't work?

[mxicat]

Nothing seems out of the ordinary. Though another small suggestion is that I don't think you need use sd-webui-infolk-prompt-comment anymore. I'm fairly certain A1111/Forge have # comments in prompts built-in now. You can search "comment" in the settings tab to make sure its enabled.

Use adetailer to inpaint an image using face_yolov8n.pt, the result is good

Which adetailer models don't work?

face_yolov8s.pt and some models from Civitai (e.g. Full Eyes detection)

[MisterChief95]

This one is a bit tricky to diagnose since it could be specific to the RTX 5000 GPUs and I unfortunately don't have one.

A couple suggestions:

1. Have you tried using a different scheduler such as sgm uniform or align your steps?
2. What happens if you uninstall the nightly torch build and instead install Pytorch 2.7 stable?

[mxicat]

This one is a bit tricky to diagnose since it could be specific to the RTX 5000 GPUs and I unfortunately don't have one.

A couple suggestions:

1. Have you tried using a different scheduler such as sgm uniform or align your steps?
2. What happens if you uninstall the nightly torch build and instead install Pytorch 2.7 stable?

1. Yes, same results.
2. I'm using 2.7.0 now and the problem is the same.

[MisterChief95]

Poked around a bit and found: https://github.com/Panchovix/stable-diffusion-webui-reForge/issues/307

================================================================================
WARNING: Official nightly torchvision packages may have compatibility issues with some extensions like ADetailer.
If you experience problems, you can replace the official package with a custom compatible version:

1. Navigate to your 'stable-diffusion-webui-reForge' folder
2. Activate the virtual environment with: venv\Scripts\activate
3. Run: pip install https://huggingface.co/Panchovix/torchvision-windows-blackwell2.0-nightly/resolve/main/torchvision-0.22.0a0%2Bd28001e-cp312-cp312-win_amd64.whl --force-reinstall
4. Restart the WebUI
================================================================================

You're using Python 3.10 so you can't use the linked torchvision, but when installing torch 2.7/nightly did you also install the newer versions of torchvision/torchaudio?