'PNDMScheduler' object has no attribute 'set_format'

[jalexs82]

Describe the bug

Getting this error from pipeline_stable_diffusion.py

Reproduction

No response

Logs

No response

System Info

On colab notebook: https://colab.research.google.com/github/WASasquatch/easydiffusion/blob/main/Stability_AI_Easy_Diffusion.ipynb

[WASasquatch]

Same issue I got. Easy Diffusion uses the pipes for Negative Weights, and they have obviously made changes to the pipe resources again, making users modified versions incompatible. Looks to be one line that's the issue. Will attempt a patch though without changes being documented by Diffusers outside GitHub automatic system, this makes reference and finding out what's changed a chore.

It's a little frustrating cause in stuff like Inpaint we have logger.info("StableDiffusionInpaintPipeline is experimental and will very likely change in the future.") Which is like OK, look out... but this is happening to all pipes a lot. Not just inpaint. Lol

[patrickvonplaten]

You are totally right - we messed it up here! Very sorry about this :pray:

Here the fix: https://github.com/huggingface/diffusers/pull/651

[WASasquatch]

I commented out the offending line of code for now, and things seem to be working without error or loss of quality.

[hafriedlander]

This (internal? unclear) API change will have also broken all custom schedulers, since most of them rely on set_format to convert from numpy to tensor. Might need a stronger warning? (I didn't see this in my console, and it took me a while to track down the issue).

[hafriedlander]

Actually, I'll suggest a different fix. Put the old behaviour back into scheduling_utils - you don't use it in the core schedulers at all, so it won't matter, and it'll mean any custom schedulers will still work.

[patrickvonplaten]

Hey @hafriedlander,

It's now a deprecated functionality but that works again. Is this good for you? Since we only support Pytorch models at the moment, I don't ever see a use case when one would need a numpy scheduler.

Does this work for you? Please let me know if it's there are some inconveniences (and which ones) with the behavior currently and whether you can transition your code to not use set_format(...)

[WASasquatch]

Hey @hafriedlander,

It's now a deprecated functionality but that works again. Is this good for you? Since we only support Pytorch models at the moment, I don't ever see a use case when one would need a numpy scheduler.

Does this work for you? Please let me know if it's there are some inconveniences (and which ones) with the behavior currently and whether you can transition your code to not use set_format(...)

Are these changes also related to this when trying to do inpaint on pndm? Can't seem to get inpaint to take any init mask in any mode.

[/usr/local/lib/python3.7/dist-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_inpaint.py](https://localhost:8080/#) in __call__(self, prompt, init_image, mask_image, strength, num_inference_steps, guidance_scale, eta, negative_prompt, generator, output_type, return_dict)
    318 
    319             # masking
--> 320             init_latents_proper = self.scheduler.add_noise(init_latents_orig, noise, t)
    321             latents = (init_latents_proper * mask) + (latents * (1 - mask))
    322 

[/usr/local/lib/python3.7/dist-packages/diffusers/schedulers/scheduling_pndm.py](https://localhost:8080/#) in add_noise(self, original_samples, noise, timesteps)
    393             self.alphas_cumprod = self.alphas_cumprod.to(original_samples.device)
    394 
--> 395         if timesteps.device != original_samples.device:
    396             timesteps = timesteps.to(original_samples.device)
    397 

AttributeError: 'numpy.int64' object has no attribute 'device'

Interestingly, running the same init mask and init image through another scheduler like lms produces

RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

[patrickvonplaten]

Hey @WASasquatch,

Would you mind opening a new issue for this?

[hafriedlander]

Hi. Sorry for delay on replay. I am using the additional schedulers Hlky put together at https://github.com/huggingface/diffusers/compare/main...hlky:diffusers:main

However I consider this solved now for two reasons:

• I just copied the old scheduling_utils.py into my server and changed my schedulers to reference that
• Hlky is going to update his schedulers to the new 0.4.0 syntax anyway, at which point he'll presumably drop the need for set_format.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.