stable-diffusion-webui-depthmap-script icon indicating copy to clipboard operation
stable-diffusion-webui-depthmap-script copied to clipboard

Published 20 hours ago • verified publisher icon thygate

dpt_beit_large_512 (midas 3.1) - ERROR

Open web2299 opened this issue 10 months ago • 8 comments

When i try to use dpt_beit_large_512 (midas 3.1) i receive this error:

ERROR: Error(s) in loading state_dict for DPTDepthModel:
Unexpected key(s) in state_dict: "pretrained.model.blocks.0.attn.relative_position_index", "pretrained.model.blocks.1.attn.relative_position_index", "pretrained.model.blocks.2.attn.relative_position_index", "pretrained.model.blocks.3.attn.relative_position_index", "pretrained.model.blocks.4.attn.relative_position_index", "pretrained.model.blocks.5.attn.relative_position_index", "pretrained.model.blocks.6.attn.relative_position_index", "pretrained.model.blocks.7.attn.relative_position_index", "pretrained.model.blocks.8.attn.relative_position_index", "pretrained.model.blocks.9.attn.relative_position_index", "pretrained.model.blocks.10.attn.relative_position_index", "pretrained.model.blocks.11.attn.relative_position_index", "pretrained.model.blocks.12.attn.relative_position_index", "pretrained.model.blocks.13.attn.relative_position_index", "pretrained.model.blocks.14.attn.relative_position_index", "pretrained.model.blocks.15.attn.relative_position_index", "pretrained.model.blocks.16.attn.relative_position_index", "pretrained.model.blocks.17.attn.relative_position_index", "pretrained.model.blocks.18.attn.relative_position_index", "pretrained.model.blocks.19.attn.relative_position_index", "pretrained.model.blocks.20.attn.relative_position_index", "pretrained.model.blocks.21.attn.relative_position_index", "pretrained.model.blocks.22.attn.relative_position_index", "pretrained.model.blocks.23.attn.relative_position_index".

Please report this issue here. Make sure to provide the full stacktrace:
Traceback (most recent call last):
  File "C:\SD_Forge\webui\extensions\stable-diffusion-webui-depthmap-script\src\common_ui.py", line 532, in run_generate
    input_i, type, result = next(gen_obj)
  File "C:\SD_Forge\webui\extensions\stable-diffusion-webui-depthmap-script\src\core.py", line 322, in core_generation_funnel
    raise e
  File "C:\SD_Forge\webui\extensions\stable-diffusion-webui-depthmap-script\src\core.py", line 124, in core_generation_funnel
    model_holder.ensure_models(inp[go.MODEL_TYPE], device, inp[go.BOOST])
  File "C:\SD_Forge\webui\extensions\stable-diffusion-webui-depthmap-script\src\depthmap_generation.py", line 62, in ensure_models
    self.load_models(model_type, device, boost)
  File "C:\SD_Forge\webui\extensions\stable-diffusion-webui-depthmap-script\src\depthmap_generation.py", line 111, in load_models
    model = DPTDepthModel(
  File "C:\SD_Forge\webui\extensions\stable-diffusion-webui-depthmap-script\dmidas\dpt_depth.py", line 163, in __init__
    self.load(path)
  File "C:\SD_Forge\webui\extensions\stable-diffusion-webui-depthmap-script\dmidas\base_model.py", line 16, in load
    self.load_state_dict(parameters)
  File "C:\SD_Forge\system\python\lib\site-packages\torch\nn\modules\module.py", line 2152, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for DPTDepthModel:
	Unexpected key(s) in state_dict: "pretrained.model.blocks.0.attn.relative_position_index", "pretrained.model.blocks.1.attn.relative_position_index", "pretrained.model.blocks.2.attn.relative_position_index", "pretrained.model.blocks.3.attn.relative_position_index", "pretrained.model.blocks.4.attn.relative_position_index", "pretrained.model.blocks.5.attn.relative_position_index", "pretrained.model.blocks.6.attn.relative_position_index", "pretrained.model.blocks.7.attn.relative_position_index", "pretrained.model.blocks.8.attn.relative_position_index", "pretrained.model.blocks.9.attn.relative_position_index", "pretrained.model.blocks.10.attn.relative_position_index", "pretrained.model.blocks.11.attn.relative_position_index", "pretrained.model.blocks.12.attn.relative_position_index", "pretrained.model.blocks.13.attn.relative_position_index", "pretrained.model.blocks.14.attn.relative_position_index", "pretrained.model.blocks.15.attn.relative_position_index", "pretrained.model.blocks.16.attn.relative_position_index", "pretrained.model.blocks.17.attn.relative_position_index", "pretrained.model.blocks.18.attn.relative_position_index", "pretrained.model.blocks.19.attn.relative_position_index", "pretrained.model.blocks.20.attn.relative_position_index", "pretrained.model.blocks.21.attn.relative_position_index", "pretrained.model.blocks.22.attn.relative_position_index", "pretrained.model.blocks.23.attn.relative_position_index". 
Time taken: 13.1 sec.

A: 0.32 GB, R: 0.34 GB, Sys: 1.5/15.9961 GB (9.3%)

I tried:

pip install timm==0.6.7

but this didnt solve the problem !

📂

web2299 avatar Mar 28 '24 08:03 web2299

same question

zzcxxoo avatar Mar 29 '24 05:03 zzcxxoo

Same issue, pip didn't fix anything.

NicoSteyl avatar Apr 02 '24 11:04 NicoSteyl

any support ?

web2299 avatar Apr 03 '24 06:04 web2299

Is this problem specific to midas?

A similar issue a while ago there was a problem with timm library. What's you version of timm?

https://github.com/thygate/stable-diffusion-webui-depthmap-script/issues/323

https://github.com/thygate/stable-diffusion-webui-depthmap-script/issues/319

graemeniedermayer avatar Apr 03 '24 07:04 graemeniedermayer

as already said i tried to install timm (many different versions) but this didnt solve the problem . Yes i think the problem is specific to midas because other models work fine .

web2299 avatar Apr 03 '24 08:04 web2299

Oh yup sorry missed that. It does look like a library issue. Like the weight file and the config are mismatched for some reason.

It's working for me.

So the file size for me is 1.5gb dpt_beit_large_512.pt (if it's different that's a problem)

Some of my versions of relevant libraries are python3.10 triton==2.1.0 torch==2.1.0+cu121 timm==0.9.2 torchvision==0.16.0

graemeniedermayer avatar Apr 03 '24 08:04 graemeniedermayer

I got it working but I don't know what did lol

In summary, I upgraded pip first. Then I opened a command shell and activated venv, I installed triton, then I launched the webui and it failed so I uninstalled triton and it looks like cmake got left behind. It's working for me now.

Powershell Log (incase a real programmer can make sense of it):

`Windows PowerShell Copyright (C) Microsoft Corporation. All rights reserved.

Install the latest PowerShell for new features and improvements! https://aka.ms/PSWindows

PS G:\AI\stable-diffusion-webui-1.8.0\venv> .\scripts\activate (venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip install triton WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) ERROR: Could not find a version that satisfies the requirement triton (from versions: none) ERROR: No matching distribution found for triton

[notice] A new release of pip is available: 22.2.1 -> 24.0 [notice] To update, run: python.exe -m pip install --upgrade pip (venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> python.exe -m pip install --upgrade pip WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) Requirement already satisfied: pip in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (22.2.1) Collecting pip Downloading pip-24.0-py3-none-any.whl.metadata (3.6 kB) Downloading pip-24.0-py3-none-any.whl (2.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 884.2 kB/s eta 0:00:00 WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) DEPRECATION: torchsde 0.2.5 has a non-standard dependency specifier numpy>=1.19.*; python_version >= "3.7". pip 24.1 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of torchsde or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063 Installing collected packages: pip Attempting uninstall: pip Found existing installation: pip 22.2.1 Uninstalling pip-22.2.1: Successfully uninstalled pip-22.2.1 Successfully installed pip-24.0 (venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip install triton WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) ERROR: Could not find a version that satisfies the requirement triton (from versions: none) ERROR: No matching distribution found for triton (venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip install https://huggingface.co/r4ziel/xformers_pre_built/resolve/main/triton-2.0.0-cp310-cp310-win_amd64.whl https://oo.pe/https://huggingface.co/r4ziel/xformers_pre_built/resolve/main/triton-2.0.0-cp310-cp310-win_amd64.whl : The term 'https://oo.pe/https://huggingface.co/r4ziel/xformers_pre_built/resolve/main/triton-2.0.0-cp310-cp310-win_amd64.whl' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. At line:1 char:116

  • ... amd64.whl](https://oo.pe/https://huggingface.co/r4ziel/xformers_pre ...
  •             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    • CategoryInfo : ObjectNotFound: (https://oo.pe/h...0-win_amd64.whl:String) [], CommandNotFoundException
    • FullyQualifiedErrorId : CommandNotFoundException

(venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip install https://huggingface.co/r4ziel/xformers_pre_built/resolve/main/triton-2.0.0-cp310-cp310-win_amd64.whl WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) Collecting triton==2.0.0 Downloading https://huggingface.co/r4ziel/xformers_pre_built/resolve/main/triton-2.0.0-cp310-cp310-win_amd64.whl (12.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.6/12.6 MB 1.1 MB/s eta 0:00:00 Collecting cmake (from triton==2.0.0) Downloading cmake-3.29.0.1-py3-none-win_amd64.whl.metadata (6.1 kB) Requirement already satisfied: filelock in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (from triton==2.0.0) (3.13.1) Requirement already satisfied: torch in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (from triton==2.0.0) (2.0.1+cu118) Requirement already satisfied: typing-extensions in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (from torch->triton==2.0.0) (4.10.0) Requirement already satisfied: sympy in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (from torch->triton==2.0.0) (1.12) Requirement already satisfied: networkx in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (from torch->triton==2.0.0) (3.2.1) Requirement already satisfied: jinja2 in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (from torch->triton==2.0.0) (3.1.2) Requirement already satisfied: MarkupSafe>=2.0 in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (from jinja2->torch->triton==2.0.0) (2.1.3) Requirement already satisfied: mpmath>=0.19 in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (from sympy->torch->triton==2.0.0) (1.3.0) Downloading cmake-3.29.0.1-py3-none-win_amd64.whl (36.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 36.2/36.2 MB 1.0 MB/s eta 0:00:00 WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) DEPRECATION: torchsde 0.2.5 has a non-standard dependency specifier numpy>=1.19.*; python_version >= "3.7". pip 24.1 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of torchsde or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063 Installing collected packages: cmake, triton Successfully installed cmake-3.29.0.1 triton-2.0.0 (venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip uninstall triton Found existing installation: triton 2.0.0 Uninstalling triton-2.0.0: Would remove: g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages\triton-2.0.0.dist-info* g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages\triton* Proceed (Y/n)? y Successfully uninstalled triton-2.0.0 (venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip cache purge Files removed: 1293 (venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip install triton-2.1.0-cp310-cp310-win_amd64.whl WARNING: Requirement 'triton-2.1.0-cp310-cp310-win_amd64.whl' looks like a filename, but the file does not exist WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) Processing g:\ai\stable-diffusion-webui-1.8.0\venv\triton-2.1.0-cp310-cp310-win_amd64.whl ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: 'G:\AI\stable-diffusion-webui-1.8.0\venv\triton-2.1.0-cp310-cp310-win_amd64.whl'

(venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip uninstall itsandbytes -y WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) WARNING: Skipping itsandbytes as it is not installed. (venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip uninstall numpy -y Found existing installation: numpy 1.26.2 Uninstalling numpy-1.26.2: ERROR: Operation cancelled by user (venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip install numpy -y

Usage: pip install [options] [package-index-options] ... pip install [options] -r [package-index-options] ... pip install [options] [-e] ... pip install [options] [-e] ... pip install [options] <archive url/path> ...

no such option: -y (venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip install numpy WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) Requirement already satisfied: numpy in g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages (1.26.2) WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) (venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> pip uninstall umpy -y WARNING: Ignoring invalid distribution -itsandbytes (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) WARNING: Ignoring invalid distribution -umpy (g:\ai\stable-diffusion-webui-1.8.0\venv\lib\site-packages) WARNING: Skipping umpy as it is not installed. (venv) PS G:\AI\stable-diffusion-webui-1.8.0\venv> `

NicoSteyl avatar Apr 03 '24 10:04 NicoSteyl

Wonderful. Oh also checking if any library has changed after auto1111 launchs is important. Sometimes there is silent requirements conflicts. Trying without any other extensions can be a helpful troubleshooting strategy (particularly newer extensions. Maybe this is a depth-anything conflict?)

graemeniedermayer avatar Apr 03 '24 16:04 graemeniedermayer