ComfyUI
ComfyUI copied to clipboard
comfyanonymous
Conv3D is not supported on MPS
failed to run SVD(stable video diffusion) on macOS
anyone can help?
Error occurred when executing KSampler: Conv3D is not supported on MPS File "/Users/ryanaltair/github/ai/ComfyUI/execution.py", line 153, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File
This may help https://github.com/pytorch/pytorch/pull/114183 and gets me past an error that looks similar to yours.
I also found that but don’t know what to do with it. 🫣 Some tip for a noobie…?
@groovybits it works but then there are other errors like [MPSNDArray initWithDevice:descriptor:] Error: total bytes of NDArray > 2**32
MPS is alpha not beta 😢
@Theodor800 I installed that pull request with: pip install git+https://github.com/pytorch/pytorch@refs/pull/114183/head. Takes A WHILE.
I get this error now on the KSampler:
`Error occurred when executing KSampler:
unsupported operand type(s) for *=: 'int' and 'NoneType'
File "/Users/jackwooldridge/StableDiffusion/ComfyUI/execution.py", line 153, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/execution.py", line 83, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/execution.py", line 76, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/nodes.py", line 1286, in sample return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/nodes.py", line 1256, in common_ksampler samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, File "/Users/jackwooldridge/StableDiffusion/ComfyUI/comfy/sample.py", line 100, in sample samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/custom_nodes/ComfyUI_smZNodes/init.py", line 129, in KSampler_sample return _KSampler_sample(*args, **kwargs) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/comfy/samplers.py", line 711, in sample return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/custom_nodes/ComfyUI_smZNodes/init.py", line 138, in sample return _sample(*args, **kwargs) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/comfy/samplers.py", line 617, in sample samples = sampler.sample(model_wrap, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/comfy/samplers.py", line 556, in sample samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/venv/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/comfy/k_diffusion/sampling.py", line 580, in sample_dpmpp_2m denoised = model(x, sigmas[i] * s_in, **extra_args) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/comfy/samplers.py", line 277, in forward out = self.inner_model(x, sigma, cond=cond, uncond=uncond, cond_scale=cond_scale, model_options=model_options, seed=seed) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in call_impl return forward_call(*args, **kwargs) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/comfy/samplers.py", line 267, in forward return self.apply_model(*args, **kwargs) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/custom_nodes/ComfyUI_smZNodes/smZNodes.py", line 892, in apply_model out = super().apply_model(*args, **kwargs) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/comfy/samplers.py", line 264, in apply_model out = sampling_function(self.inner_model, x, timestep, uncond, cond, cond_scale, model_options=model_options, seed=seed) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/comfy/samplers.py", line 252, in sampling_function cond, uncond = calc_cond_uncond_batch(model, cond, uncond, x, timestep, model_options) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/comfy/samplers.py", line 230, in calc_cond_uncond_batch output = model.apply_model(input_x, timestep, **c).chunk(batch_chunks) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/custom_nodes/ComfyUI_smZNodes/modules/sd_hijack_utils.py", line 17, in setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs)) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/custom_nodes/ComfyUI_smZNodes/modules/sd_hijack_utils.py", line 28, in call return self.__orig_func(*args, **kwargs) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/comfy/model_base.py", line 73, in apply_model model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float() File "/Users/jackwooldridge/StableDiffusion/ComfyUI/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/custom_nodes/FreeU_Advanced/nodes.py", line 173, in __temp__forward h = forward_timestep_embed(module, h, emb, context, transformer_options) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/comfy/ldm/modules/diffusionmodules/openaimodel.py", line 38, in forward_timestep_embed x = layer(x, emb, num_video_frames, image_only_indicator) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/comfy/ldm/modules/diffusionmodules/openaimodel.py", line 337, in forward x_mix = rearrange(x, "(b t) c h w -> b c t h w", t=num_video_frames) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/venv/lib/python3.9/site-packages/einops/einops.py", line 591, in rearrange return reduce(tensor, pattern, reduction="rearrange", **axes_lengths) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/venv/lib/python3.9/site-packages/einops/einops.py", line 523, in reduce return _apply_recipe( File "/Users/jackwooldridge/StableDiffusion/ComfyUI/venv/lib/python3.9/site-packages/einops/einops.py", line 239, in _apply_recipe _result = _reconstruct_from_shape_uncached(recipe, backend.shape(tensor), axes_lengths) File "/Users/jackwooldridge/StableDiffusion/ComfyUI/venv/lib/python3.9/site-packages/einops/einops.py", line 179, in _reconstruct_from_shape_uncached known_product *= axes_lengths[axis]`
@jwooldridge234 Thank you. Have to try that soon. I had also that operand type error, but you'll solve that by uninstalling FreeU_Advanced node.
@Theodor800 Thanks- that solved that error. Now just crashes on render (running the first img2vid workflow from the examples):
Requested to load SVD_img2vid Loading 1 new model 0%| | 0/20 [00:00<?, ?it/s]/Users/jackwooldridge/miniconda3/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d ' zsh: segmentation fault python main.py
@jwooldridge234 Errors after errors everyday. 😀 I hope you find solution for that one. I got mine to work now. So thank you very much! And thanks @groovybits!
This may help pytorch/pytorch#114183 and gets me past an error that looks similar to yours.
I'm working on that PR. I found an issue in the Conv3D implementation and I need to debug it and write tests for it before making the PR as ready for review.
Try something like:
pip install git+https://github.com/pytorch/pytorch@refs/pull/114183/head git+https://github.com/pytorch/vision git+https://github.com/pytorch/audio
At least until @LucasSte 's PR is merged.
Error occurred when executing KSampler:
Conv3D is not supported on MPS
Hi All, anyone able to solve this one please.
I dont know if this helps but I just found it https://github.com/cocktailpeanut/mac-svd-install
I tried pip install git+https://github.com/pytorch/pytorch@refs/pull/114183/head git+https://github.com/pytorch/vision git+https://github.com/pytorch/audio
Its taking almost 2 hours and showing below command
Preparing metadata (pyproject.toml) ... |
Finally end up with below error.
Resolved https://github.com/pytorch/audio to commit ccd78ffba3208b4a9f5face4bd76e942ddc2d4a3 Running command git submodule update --init --recursive -q Installing build dependencies ... done Getting requirements to build wheel ... error error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [17 lines of output]
Traceback (most recent call last):
File "/opt/anaconda3/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in
note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
Pytorch issue. I had similar error and I couldnt even run comfyUI anymore after trying it so had to uninstall torch and re-install. Comfy works now but still cannot get SVD to run. Probably best to wait till its resolved.
@thekannanmca i'm also experiencing the same exact error. please let me know if you figure out a workaround
A workaround would be using the code from my PR (https://github.com/pytorch/pytorch/pull/114183). You would need to clone the repository, fetch my PR's changes and build PyTorch from scratch. This is a bit time consuming.
git clone --recursive https://github.com/pytorch/pytorch
cd pytorch
git submodule update --init --recursive
git fetch origin pull/114183/head:conv-3d-mps
git checkout conv-3d-mps
python3 setup.py develop
The core issue encountered was that the Conv3D operation is currently not supported on Apple's Metal Performance Shaders (MPS) through PyTorch. To utilize Conv3D with MPS, we must await an update from PyTorch that extends support for this functionality. In the interim, I devised a workaround to circumvent this limitation. The modification was made in the model_management.py file within the ComfyUI directory:
The original function get_torch_device() contained logic to determine and return the appropriate compute device based on the current environment's capabilities:
def get_torch_device(): global directml_enabled global cpu_state if directml_enabled: global directml_device return directml_device if cpu_state == CPUState.MPS: return torch.device("mps") if cpu_state == CPUState.CPU: return torch.device("cpu") else: if is_intel_xpu(): return torch.device("xpu") else: return torch.device(torch.cuda.current_device()) This was simplified to exclusively direct PyTorch to utilize the CPU for all operations:
def get_torch_device(): return torch.device("cpu")
By applying this change, the Conv3D operation and other computations are forced to run on the CPU. While this resolves the immediate compatibility issue, it's important to note that computation will be slower due to the CPU's comparative performance limitations. The model will function correctly, albeit with increased processing times for generating the output.
This solution is a temporary measure until PyTorch introduces MPS support for the Conv3D operation, after which full hardware acceleration can be leveraged for improved efficiency.
A workaround would be using the code from my PR (pytorch/pytorch#114183). You would need to clone the repository, fetch my PR's changes and build PyTorch from scratch. This is a bit time consuming.
git clone --recursive https://github.com/pytorch/pytorch cd pytorch git submodule update --init --recursive git fetch origin pull/114183/head:conv-3d-mps git checkout conv-3d-mps python3 setup.py develop
I'm getting a segmentation fault 11 once it gets to the ksampler (Running on a mac m2 with 24G). I also saw that someone running on an M1 Ultra studio has the same issue. https://github.com/Stability-AI/StableSwarmUI/issues/185#issuecomment-1829550617.
@jwooldridge234 I have the same error that you had last week.
UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '
Did you ever find a solution?
@mynameisbill Afraid not- going to wait until pytorch gets patched to get this working locally.
As of now I am using AnimatedDiff, I am hoping for the best for this issue to be resolved
Bumping as this appears to have been fixed in pytorch #https://github.com/pytorch/pytorch/pull/114183
FWIW I just tried running SVD using ComfyUI with the latest PyTorch nightly that includes Conv3D support on a 16gb M1.
I am no longer getting the Conv3D not supported error, but I am running out of memory:
RuntimeError: MPS backend out of memory (MPS allocated: 8.39 GB, other allocations: 9.40 GB, max allowed: 18.13 GB). Tried to allocate 420.00 MB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
Prompt executed in 89.71 seconds
I ran ComfyUI using python main.py --force-fp16 and was using the svd-fp16.safetensors checkpoint from https://huggingface.co/becausecurious/stable-video-diffusion-img2vid-fp16.
Running ComfyUI with SVD XT on MBP M1 32GB.
Installed pytorch versions: torch 2.2.0a0+gitc9950e1 torchaudio 0.13.1 torchvision 0.14.1
Using the --force-fp16 option, and the svd_xt.safetensors
Although it's the 25 frames model, I too get RuntimeError: MPS backend out of memory even while using euler_ancestral sampler and karras scheduler.
The only way I get it to work without errors is to configure the conditioning node to video_frames: 14 instead of 25.
At least it's working and I can learn to use it, until 25 frames will fit into 32GB...
Still, it's painfully slow:
100%|███████████████████████████████████████████| 20/20 [22:52<00:00, 68.60s/it]
@KarthikDani haven't tried with the M1 Air 8GB, I believe there's not enough RAM, and with the pro it takes over 20 minutes so yeah...
The core issue encountered was that the Conv3D operation is currently not supported on Apple's Metal Performance Shaders (MPS) through PyTorch. To utilize Conv3D with MPS, we must await an update from PyTorch that extends support for this functionality. In the interim, I devised a workaround to circumvent this limitation. The modification was made in the model_management.py file within the ComfyUI directory:
The original function get_torch_device() contained logic to determine and return the appropriate compute device based on the current environment's capabilities:
def get_torch_device(): global directml_enabled global cpu_state if directml_enabled: global directml_device return directml_device if cpu_state == CPUState.MPS: return torch.device("mps") if cpu_state == CPUState.CPU: return torch.device("cpu") else: if is_intel_xpu(): return torch.device("xpu") else: return torch.device(torch.cuda.current_device()) This was simplified to exclusively direct PyTorch to utilize the CPU for all operations:
def get_torch_device(): return torch.device("cpu")
By applying this change, the Conv3D operation and other computations are forced to run on the CPU. While this resolves the immediate compatibility issue, it's important to note that computation will be slower due to the CPU's comparative performance limitations. The model will function correctly, albeit with increased processing times for generating the output.
This solution is a temporary measure until PyTorch introduces MPS support for the Conv3D operation, after which full hardware acceleration can be leveraged for improved efficiency.
It works for me but Too slow to apply it...
