ComfyUI icon indicating copy to clipboard operation
ComfyUI copied to clipboard
Published 4 days ago • verified publisher icon comfyanonymous

convolution_overrideable not implemented when executing VAEEncode

Open spaetische opened this issue 1 year ago • 3 comments

Expected Behavior

Use Inpaint Preprocessor with Load Checkpoint (Realistic Vision V5.1 Inpainting) and ControlNet Model (control_v11p_sd15_inpaint) to inpaint a masked area of a loaded image based on a text prompt. Screenshot 2024-08-04 at 23 24 15

Actual Behavior

Whenever the process reaches VAE Encode node, the error message emerges. Screenshot 2024-08-04 at 23 24 26

Steps to Reproduce

  1. Launch ComfyUI on MacBook Air M3.
  2. Select Realistic Vision model v5.1 model in the Load Checkpoint node.
  3. Select ControlNet inpaint model in the Load ControlNet Model node.
  4. Load an image and create a mask in MaskEditor.
  5. Queue prompt.

Debug Logs

"!!! Exception during processing!!! convolution_overrideable not implemented. You are likely triggering this with tensor backend other than CPU/CUDA/MKLDNN, if this is intended, please use TORCH_LIBRARY_IMPL to override this function 
Traceback (most recent call last):
  File "/Users/yegorartyukh/ComfyUI/execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/yegorartyukh/ComfyUI/execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/yegorartyukh/ComfyUI/execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/yegorartyukh/ComfyUI/nodes.py", line 296, in encode
    t = vae.encode(pixels[:,:,:,:3])
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/yegorartyukh/ComfyUI/comfy/sd.py", line 346, in encode
    samples[x:x+batch_number] = self.first_stage_model.encode(pixels_in).to(self.output_device).float()
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/yegorartyukh/ComfyUI/comfy/ldm/models/autoencoder.py", line 179, in encode
    z = self.encoder(x)
        ^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/yegorartyukh/ComfyUI/comfy/ldm/modules/diffusionmodules/model.py", line 520, in forward
    h = self.conv_in(x)
        ^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/yegorartyukh/ComfyUI/comfy/ops.py", line 93, in forward
    return super().forward(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 549, in forward
    return self._conv_forward(input, self.weight, self.bias)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 544, in _conv_forward
    return F.conv2d(
           ^^^^^^^^^
NotImplementedError: convolution_overrideable not implemented. You are likely triggering this with tensor backend other than CPU/CUDA/MKLDNN, if this is intended, please use TORCH_LIBRARY_IMPL to override this function"

Other

I was trying different solutions:

  1. Python versions 3.10, 3.11, 3.12.
  2. Different PyTorch versions.
  3. Setting a device explicitly to mps and/or CPU in the main script.

MacBook Air M3 2024, 16GB, macOS Sonoma 14.3, Python 3.11, PyTorch 2.5 Preview (Nightly).

spaetische avatar Aug 04 '24 21:08 spaetische

Not sure if this belongs here or on the SwarmUI repo (error seems to be coming from ComfyUI), but I'm getting the same error on a Macbook pro M3 Max. Happens when I enable tiling on the refiner stage in Swarm. Just like spaetische, I've tried multiple versions of PyTorch and I get the same error.

Here are some logs:

15:43:03.950 [Warning] [ComfyUI-0/STDERR] Traceback (most recent call last):
15:43:04.080 [Warning] [ComfyUI-0/STDERR]   File "/Users/some_user/Developer/ai-projects/SwarmUI/dlbackend/ComfyUI/execution.py", line 152, in recursive_execute
15:43:04.110 [Warning] [ComfyUI-0/STDERR]     output_data, output_ui = get_output_data(obj, input_data_all)
15:43:04.110 [Warning] [ComfyUI-0/STDERR]                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15:43:04.110 [Warning] [ComfyUI-0/STDERR]   File "/Users/some_user/Developer/ai-projects/SwarmUI/dlbackend/ComfyUI/execution.py", line 82, in get_output_data
15:43:04.112 [Warning] [ComfyUI-0/STDERR]     return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
15:43:04.112 [Warning] [ComfyUI-0/STDERR]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15:43:04.112 [Warning] [ComfyUI-0/STDERR]   File "/Users/some_user/Developer/ai-projects/SwarmUI/dlbackend/ComfyUI/execution.py", line 75, in map_node_over_list
15:43:04.112 [Warning] [ComfyUI-0/STDERR]     results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
15:43:04.118 [Warning] [ComfyUI-0/STDERR]                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15:43:04.118 [Warning] [ComfyUI-0/STDERR]   File "/Users/some_user/Developer/ai-projects/SwarmUI/dlbackend/ComfyUI/nodes.py", line 270, in decode
15:43:04.118 [Warning] [ComfyUI-0/STDERR]     return (vae.decode(samples["samples"]), )
15:43:04.119 [Warning] [ComfyUI-0/STDERR]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15:43:04.119 [Warning] [ComfyUI-0/STDERR]   File "/Users/some_user/Developer/ai-projects/SwarmUI/dlbackend/ComfyUI/comfy/sd.py", line 318, in decode
15:43:04.119 [Warning] [ComfyUI-0/STDERR]     pixel_samples[x:x+batch_number] = self.process_output(self.first_stage_model.decode(samples).to(self.output_device).float())
15:43:04.119 [Warning] [ComfyUI-0/STDERR]                                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15:43:04.119 [Warning] [ComfyUI-0/STDERR]   File "/Users/some_user/Developer/ai-projects/SwarmUI/dlbackend/ComfyUI/comfy/ldm/models/autoencoder.py", line 200, in decode
15:43:04.119 [Warning] [ComfyUI-0/STDERR]     dec = self.decoder(dec, **decoder_kwargs)
15:43:04.119 [Warning] [ComfyUI-0/STDERR]           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15:43:04.119 [Warning] [ComfyUI-0/STDERR]   File "/Users/some_user/Developer/ai-projects/SwarmUI/dlbackend/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
15:43:04.119 [Warning] [ComfyUI-0/STDERR]     return self._call_impl(*args, **kwargs)
15:43:04.119 [Warning] [ComfyUI-0/STDERR]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15:43:04.119 [Warning] [ComfyUI-0/STDERR]   File "/Users/some_user/Developer/ai-projects/SwarmUI/dlbackend/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
15:43:04.120 [Warning] [ComfyUI-0/STDERR]     return forward_call(*args, **kwargs)
15:43:04.120 [Warning] [ComfyUI-0/STDERR]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15:43:04.122 [Warning] [ComfyUI-0/STDERR]   File "/Users/some_user/Developer/ai-projects/SwarmUI/dlbackend/ComfyUI/comfy/ldm/modules/diffusionmodules/model.py", line 639, in forward
15:43:04.123 [Warning] [ComfyUI-0/STDERR]     h = self.up[i_level].upsample(h)
15:43:04.123 [Warning] [ComfyUI-0/STDERR]         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15:43:04.123 [Warning] [ComfyUI-0/STDERR]   File "/Users/some_user/Developer/ai-projects/SwarmUI/dlbackend/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
15:43:04.123 [Warning] [ComfyUI-0/STDERR]     return self._call_impl(*args, **kwargs)
15:43:04.123 [Warning] [ComfyUI-0/STDERR]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15:43:04.123 [Warning] [ComfyUI-0/STDERR]   File "/Users/some_user/Developer/ai-projects/SwarmUI/dlbackend/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
15:43:04.123 [Warning] [ComfyUI-0/STDERR]     return forward_call(*args, **kwargs)
15:43:04.123 [Warning] [ComfyUI-0/STDERR]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15:43:04.123 [Warning] [ComfyUI-0/STDERR]   File "/Users/some_user/Developer/ai-projects/SwarmUI/dlbackend/ComfyUI/comfy/ldm/modules/diffusionmodules/model.py", line 72, in forward
15:43:04.123 [Warning] [ComfyUI-0/STDERR]     x = self.conv(x)
15:43:04.123 [Warning] [ComfyUI-0/STDERR]         ^^^^^^^^^^^^
15:43:04.123 [Warning] [ComfyUI-0/STDERR]   File "/Users/some_user/Developer/ai-projects/SwarmUI/dlbackend/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
15:43:04.123 [Warning] [ComfyUI-0/STDERR]     return self._call_impl(*args, **kwargs)
15:43:04.123 [Warning] [ComfyUI-0/STDERR]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15:43:04.123 [Warning] [ComfyUI-0/STDERR]   File "/Users/some_user/Developer/ai-projects/SwarmUI/dlbackend/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
15:43:04.123 [Warning] [ComfyUI-0/STDERR]     return forward_call(*args, **kwargs)
15:43:04.123 [Warning] [ComfyUI-0/STDERR]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15:43:04.123 [Warning] [ComfyUI-0/STDERR]   File "/Users/some_user/Developer/ai-projects/SwarmUI/dlbackend/ComfyUI/comfy/ops.py", line 93, in forward
15:43:04.123 [Warning] [ComfyUI-0/STDERR]     return super().forward(*args, **kwargs)
15:43:04.123 [Warning] [ComfyUI-0/STDERR]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15:43:04.123 [Warning] [ComfyUI-0/STDERR]   File "/Users/some_user/Developer/ai-projects/SwarmUI/dlbackend/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 460, in forward
15:43:04.123 [Warning] [ComfyUI-0/STDERR]     return self._conv_forward(input, self.weight, self.bias)
15:43:04.123 [Warning] [ComfyUI-0/STDERR]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15:43:04.124 [Warning] [ComfyUI-0/STDERR]   File "/Users/some_user/Developer/ai-projects/SwarmUI/dlbackend/ComfyUI/venv/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 456, in _conv_forward
15:43:04.124 [Warning] [ComfyUI-0/STDERR]     return F.conv2d(input, weight, bias, self.stride,
15:43:04.124 [Warning] [ComfyUI-0/STDERR]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
15:43:04.124 [Warning] [ComfyUI-0/STDERR] NotImplementedError: convolution_overrideable not implemented. You are likely triggering this with tensor backend other than CPU/CUDA/MKLDNN, if this is intended, please use TORCH_LIBRARY_IMPL to override this function
15:43:04.124 [Warning] [ComfyUI-0/STDERR]

salinas707 avatar Aug 06 '24 01:08 salinas707

were you able to get this resolved? I'm still getting that error on my M3 max... I've tried different versions of pytorch and it doesn't make any difference

salinas707 avatar Aug 08 '24 18:08 salinas707

were you able to get this resolved? I'm still getting that error on my M3 max... I've tried different versions of pytorch and it doesn't make any difference

Nope. No solution so far. I guess it's just a classic software limitation/bug that a user with mac OS gets

spaetische avatar Aug 09 '24 11:08 spaetische

Have you guys solved this problem? I've encountered it too.

18YuYongjie avatar Feb 06 '25 15:02 18YuYongjie

I have downscaled the image before connecting it to vaencoder. Hope it works

delsotano avatar Mar 22 '25 22:03 delsotano