[Bug]: RuntimeError: Placeholder storage has not been allocated on MPS device!

[EdgarNoob]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

SD works fine, but after enabling ControlNet v1.1.130 it crashes with RuntimeError: Placeholder storage has not been allocated on MPS device!

P.S Deforum crashes too. even if ControlNet is not enabled.

I am n00b, that means, if you can help, but you need more info, PLEASE ASK :)

Steps to reproduce the problem

1. Go to ....
2. Press ....
3. ...

What should have happened?

It should generate image.

Commit where the problem happens

webui: controlnet:

What browsers do you use to access the UI ?

No response

Command Line Arguments

I am launching SD with ./webui.sh

Console logs

Error completing request
Arguments: ('task(5x96ui6r1vjgx1r)', 0, 'anime ', '', [], <PIL.Image.Image image mode=RGBA size=1080x1920 at 0x38CDD4640>, None, None, None, None, None, None, 20, 0, 4, 0, 1, False, False, 1, 1, 21.5, 1.5, 0.26, -1.0, -1.0, 0, 0, 0, False, 0, 960, 540, 1, 0, 0, 32, 0, '', '', '', [], 0, <controlnet.py.UiControlNetUnit object at 0x38cdd7490>, <controlnet.py.UiControlNetUnit object at 0x38cdd70d0>, <controlnet.py.UiControlNetUnit object at 0x38cdd7fa0>, '<ul>\n<li><code>CFG Scale</code> should be 2 or lower.</li>\n</ul>\n', True, True, '', '', True, 50, True, 1, 0, False, 4, 0.5, 'Linear', 'None', '<p style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', 0, False, False, '', '<p style="margin-bottom:0.75em">Will upscale the image by the selected scale factor; use width and height sliders to set tile size</p>', 64, 0, 2, 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, None, False, None, False, None, False, 50) {}
Traceback (most recent call last):
  File "/Users/edgars/stable-diffusion-webui/modules/call_queue.py", line 57, in f
    res = list(func(*args, **kwargs))
  File "/Users/edgars/stable-diffusion-webui/modules/call_queue.py", line 37, in f
    res = func(*args, **kwargs)
  File "/Users/edgars/stable-diffusion-webui/modules/img2img.py", line 181, in img2img
    processed = process_images(p)
  File "/Users/edgars/stable-diffusion-webui/modules/processing.py", line 515, in process_images
    res = process_images_inner(p)
  File "/Users/edgars/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/batch_hijack.py", line 42, in processing_process_images_hijack
    return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
  File "/Users/edgars/stable-diffusion-webui/modules/processing.py", line 669, in process_images_inner
    samples_ddim = p.sample(conditioning=c, unconditional_conditioning=uc, seeds=seeds, subseeds=subseeds, subseed_strength=p.subseed_strength, prompts=prompts)
  File "/Users/edgars/stable-diffusion-webui/modules/processing.py", line 1115, in sample
    samples = self.sampler.sample_img2img(self, self.init_latent, x, conditioning, unconditional_conditioning, image_conditioning=self.image_conditioning)
  File "/Users/edgars/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 350, in sample_img2img
    samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
  File "/Users/edgars/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 251, in launch_sampling
    return func()
  File "/Users/edgars/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 350, in <lambda>
    samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
  File "/Users/edgars/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/Users/edgars/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/sampling.py", line 145, in sample_euler_ancestral
    denoised = model(x, sigmas[i] * s_in, **extra_args)
  File "/Users/edgars/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/edgars/stable-diffusion-webui/modules/sd_samplers_kdiffusion.py", line 141, in forward
    x_out[a:b] = self.inner_model(x_in[a:b], sigma_in[a:b], cond=make_condition_dict([cond_in[a:b]], image_cond_in[a:b]))
  File "/Users/edgars/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/edgars/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/external.py", line 114, in forward
    eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
  File "/Users/edgars/stable-diffusion-webui/repositories/k-diffusion/k_diffusion/external.py", line 140, in get_eps
    return self.inner_model.apply_model(*args, **kwargs)
  File "/Users/edgars/stable-diffusion-webui/modules/sd_hijack_utils.py", line 17, in <lambda>
    setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
  File "/Users/edgars/stable-diffusion-webui/modules/sd_hijack_utils.py", line 28, in __call__
    return self.__orig_func(*args, **kwargs)
  File "/Users/edgars/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 858, in apply_model
    x_recon = self.model(x_noisy, t, **cond)
  File "/Users/edgars/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1538, in _call_impl
    result = forward_call(*args, **kwargs)
  File "/Users/edgars/stable-diffusion-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 1335, in forward
    out = self.diffusion_model(x, t, context=cc)
  File "/Users/edgars/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/edgars/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/hook.py", line 274, in forward2
    return forward(*args, **kwargs)
  File "/Users/edgars/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/hook.py", line 205, in forward
    control = param.control_model(x=x_in, hint=param.used_hint_cond, timesteps=timesteps, context=context)
  File "/Users/edgars/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/edgars/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/cldm.py", line 99, in forward
    return self.control_model(*args, **kwargs)
  File "/Users/edgars/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/edgars/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/cldm.py", line 352, in forward
    emb = self.time_embed(t_emb)
  File "/Users/edgars/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/edgars/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/container.py", line 217, in forward
    input = module(input)
  File "/Users/edgars/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Users/edgars/stable-diffusion-webui/extensions-builtin/Lora/lora.py", line 311, in lora_Linear_forward
    return torch.nn.Linear_forward_before_lora(self, input)
  File "/Users/edgars/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 114, in forward
    return F.linear(input, self.weight, self.bias)
RuntimeError: Placeholder storage has not been allocated on MPS device!

Additional information

No response

[littleyeson]

I have a same problem in Mac M1

[littleyeson]

I fixed it. In webui.sh or webui-user.sh del --upcast-sampling --use-cpu. Change "--no-half-vae" to "--no-half".Controne net can work.

[newledge]

I fixed it. In webui.sh or webui-user.sh del --upcast-sampling --use-cpu. Change "--no-half-vae" to "--no-half".Controne net can work.

Mac M1

torch 2.0.1 torchdiffeq 0.2.3 torchmetrics 0.11.4 torchsde 0.2.5 torchvision 0.15.2

python3.10.11

ControlNet v1.1.150

webui-user.sh export COMMANDLINE_ARGS="--skip-torch-cuda-test --no-half --use-cpu all"

error: RuntimeError: Placeholder storage has not been allocated on MPS device!

change ： " del --upcast-sampling --use-cpu. Change "--no-half-vae" to "--no-half" can not start webui.sh

[EdgarNoob]

del --upcast-sampling --use-cpu. Change "--no-ha

Been there done that...doesn't work anyways, but good for you :)

[newledge]

del --upcast-sampling --use-cpu. Change "--no-ha

Been there done that...doesn't work anyways, but good for you :)

How to fix this error?

[korronald]

Same problem here. Different combinations and different kinds of errors with ControlNet on mac M1

[scythargon]

Guys, hello! Having the same problem on M1, even updated my MacOS to v13, tried different torch versions and different COMMANDLINE_ARGS T_T Help please.

UPD: ughhhh, after few hours of digging, this group of parameters has worked finally: --skip-torch-cuda-test --no-half (Remove the rest of them)

[aryanbhasin]

Guys, hello! Having the same problem on M1, even updated my MacOS to v13, tried different torch versions and different COMMANDLINE_ARGS T_T Help please.

UPD: ughhhh, after few hours of digging, this group of parameters has worked finally: --skip-torch-cuda-test --no-half (Remove the rest of them)

This worked for me 🙂

[wastu01]

--skip-torch-cuda-test --no-half (Remove the rest of them)

Have same problem on M1 Mac, fixed it by this, tks