sd-webui-controlnet icon indicating copy to clipboard operation
sd-webui-controlnet copied to clipboard

[Bug]: depth zoe cannot run normally under torch2.1 and torch2.2

Open RunawayOrange opened this issue 1 year ago • 2 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

其他预处理器能正常运行,但唯独depth zoe在torch2.1和torch2.2环境下均无法正常运行,但depth zoe可以在torch2.0环境下正常运行

Steps to reproduce the problem

在torch2.0下可正常运行 torch2.1和torch2.2环境下均无法正常运行

What should have happened?

应该像其他预处理器一样正常运行

Commit where the problem happens

webui: v1.7.0-437-gce168ab5 controlnet: 1.1.440

What browsers do you use to access the UI ?

No response

Command Line Arguments

--medvram-sdxl --theme dark --xformers --api --autolaunch

List of enabled extensions

AAA

Console logs

img_size [384, 512]
Params passed to Resize transform:
        width:  512
        height:  384
        resize_target:  True
        keep_aspect_ratio:  True
        ensure_multiple_of:  32
        resize_method:  minimal
Traceback (most recent call last):
  File "F:\Stable-Diffusion\venv\Lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\Stable-Diffusion\venv\Lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\Stable-Diffusion\venv\Lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\Stable-Diffusion\venv\Lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\Stable-Diffusion\venv\Lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "F:\Stable-Diffusion\venv\Lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\Stable-Diffusion\venv\Lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^
  File "F:\Stable-Diffusion\extensions\sd-webui-controlnet\scripts\controlnet_ui\controlnet_ui_group.py", line 1013, in run_annotator
    result, is_image = preprocessor(
                       ^^^^^^^^^^^^^
  File "F:\Stable-Diffusion\extensions\sd-webui-controlnet\scripts\utils.py", line 80, in decorated_func
    return cached_func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\Stable-Diffusion\extensions\sd-webui-controlnet\scripts\utils.py", line 64, in cached_func
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "F:\Stable-Diffusion\extensions\sd-webui-controlnet\scripts\global_state.py", line 37, in unified_preprocessor
    return preprocessor_modules[preprocessor_name](*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\Stable-Diffusion\extensions\sd-webui-controlnet\scripts\processor.py", line 558, in zoe_depth
    result = model_zoe_depth(img)
             ^^^^^^^^^^^^^^^^^^^^
  File "F:\Stable-Diffusion\extensions\sd-webui-controlnet\annotator\zoe\__init__.py", line 38, in __call__
    self.load_model()
  File "F:\Stable-Diffusion\extensions\sd-webui-controlnet\annotator\zoe\__init__.py", line 28, in load_model
    model.load_state_dict(torch.load(modelpath, map_location=model.device)['model'])
  File "F:\Stable-Diffusion\venv\Lib\site-packages\torch\nn\modules\module.py", line 2153, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for ZoeDepth:
        Unexpected key(s) in state_dict: "core.core.pretrained.model.blocks.0.attn.relative_position_index", "core.core.pretrained.model.blocks.1.attn.relative_position_index", "core.core.pretrained.model.blocks.2.attn.relative_position_index", "core.core.pretrained.model.blocks.3.attn.relative_position_index", "core.core.pretrained.model.blocks.4.attn.relative_position_index", "core.core.pretrained.model.blocks.5.attn.relative_position_index", "core.core.pretrained.model.blocks.6.attn.relative_position_index", "core.core.pretrained.model.blocks.7.attn.relative_position_index", "core.core.pretrained.model.blocks.8.attn.relative_position_index", "core.core.pretrained.model.blocks.9.attn.relative_position_index", "core.core.pretrained.model.blocks.10.attn.relative_position_index", "core.core.pretrained.model.blocks.11.attn.relative_position_index", "core.core.pretrained.model.blocks.12.attn.relative_position_index", "core.core.pretrained.model.blocks.13.attn.relative_position_index", "core.core.pretrained.model.blocks.14.attn.relative_position_index", "core.core.pretrained.model.blocks.15.attn.relative_position_index", "core.core.pretrained.model.blocks.16.attn.relative_position_index", "core.core.pretrained.model.blocks.17.attn.relative_position_index", "core.core.pretrained.model.blocks.18.attn.relative_position_index", "core.core.pretrained.model.blocks.19.attn.relative_position_index", "core.core.pretrained.model.blocks.20.attn.relative_position_index", "core.core.pretrained.model.blocks.21.attn.relative_position_index", "core.core.pretrained.model.blocks.22.attn.relative_position_index", "core.core.pretrained.model.blocks.23.attn.relative_position_index".

Additional information

No response

RunawayOrange avatar Jan 31 '24 15:01 RunawayOrange

Same

light-and-ray avatar Mar 21 '24 07:03 light-and-ray

Same

waterdrop1024 avatar Apr 10 '24 06:04 waterdrop1024

run pip install timm==0.6.7 and restart webui solved my problem

sparkleMing avatar May 20 '24 07:05 sparkleMing

pip install timm==0.9.5 This version also solves the problem

hben35096 avatar May 22 '24 17:05 hben35096

@huchenlei Detection seems to be a bit of a problem, I've installed timm==0.9.5 but still prompts:

image

image

The writing may need to be adjusted: image

hben35096 avatar May 22 '24 21:05 hben35096

@huchenlei It worked, thanks ;-)

hben35096 avatar May 22 '24 21:05 hben35096