[Bug]: reference_only is crashing

[FrancisBourre]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

I cannot use reference_only. Tried many combinations, crashed every time.

Steps to reproduce the problem

1. Enable controlnet on txt2img tab
2. Drop an image
3. Select reference_only in the combo box
4. Click on generate

What should have happened?

Reference Only should not throw an exception and crash the web ui.

Commit where the problem happens

webui: Version: v1.2.1 Commit hash: 89f9faa63388756314e8a1d96cf86bf5e0663045 controlnet: Version 1.1.173 d831043cb81e97724ccf9f071da391d479440a77

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

--skip-torch-cuda-test --no-half

Console logs

Loading preprocessor: reference_only
preprocessor resolution = 512
  0%|                                                                                                                                                         | 0/20 [00:00<?, ?it/s]ControlNet used torch.float32 VAE to encode torch.Size([2, 4, 64, 64]).
/AppleInternal/Library/BuildRoots/9941690d-bcf7-11ed-a645-863efbbaf80d/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Types/MPSNDArray.mm:706: failed assertion `[MPSTemporaryNDArray initWithDevice:descriptor:] Error: NDArray dimension length > INT_MAX'
zsh: abort      ./webui.sh

Additional information

I'm using a mac M2 with 96GB ram.

[shawnington]

Also crashing on M1 Max after updating with the error:

failed assertion '[MPSTemporaryNDArray initWithDevice:descriptor:] Error: NDArray dimension length > INT_MAX'

-only crashing for reference_only all other modules appear to be working correctly for controlNet -only occurs when Upcast cross attention layer to float32 is selected in Stable Diffiusion settings

Im guessing it is related to upcasting to float32 not being supported for reference_only mode in some way.

[gregorio500]

same here. I've already updated, but it doesn't help with my Mac M1. When I try to use 'reference only' in ControlNet, it throws the error: "Python quit unexpectedly." I've tried using several models, but the same error occur. More details about this error:

Error: NDArray dimension length > INT_MAX'
zsh: abort      ./webui.sh --no-half --disable-safe-unpickle
(base) gregorio@greg-MacBook-Pro stable-diffusion-webui % /opt/homebrew/Cellar/[email protected]/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown

[mynameisbill]

Same story here. Running on M1 Ultra with 128 GB Ram. Python quits every time I run reference_only or reference_adain+attn. Curiously, reference_adain works just fine.

[rainx]

the same issue, I am using Apple M1 Pro with 32RAM. Python 3.10.10

[rootlake]

Confirmed here as well on an M1 Pro. Reference_adain works, the other reference models do not.

[mynameisbill]

as @shawnington pointed out, it does work when "Upcast cross attention layer to float32" is switched off in Stable Diffusion settings. I did that, used it for awhile, and then toggled the setting back on. And now it seems to work fine. No idea why.

[gregorio500]

The solution to switch off "Upcast cross attention layer to float32" does not work for me.

[firestar222]

Same for me, and "Upcast cross attention layer to float32" does not work for me either... M2 Max

[er1cw00]

the same issue on Mac M1 Pro + RAM 32G :

ControlNet used torch.float32 VAE to encode torch.Size([2, 4, 64, 64]).
/AppleInternal/Library/BuildRoots/9941690d-bcf7-11ed-a645-863efbbaf80d/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Types/MPSNDArray.mm:706: failed assertion `[MPSTemporaryNDArray initWithDevice:descriptor:] Error: NDArray dimension length > INT_MAX'
zsh: abort      ./webui.sh --no-half
(/Users/wadahana/workspace/AI/conda) wadahana@wuxindeMBP stable-diffusion-webui % /Users/wadahana/workspace/AI/conda/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

[Zaiwa-linus]

This issue might potentially be due to certain parts of the numpy library used in reference_only not being installable without administrator privileges, although this is not definitively confirmed.

I've been able to resolve the issue using this method and have verified its operation on an M1 Pro environment with 32GB memory.

The problem should be resolved by using administrator privileges to install numpy in a virtual environment, as shown in the commands below:

. venv/bin/activate
pip install --upgrade pip
pip install torch torchvision torchaudio numpy --upgrade --ignore-installed
sudo pip install numpy --upgrade --ignore-installed
./webui.sh

I hope this post can be of help to you.