EulerDiscreteScheduler.set_timesteps() torch.from_numpy misuse error.

[Teriks]

Describe the bug

In version 0.24.0, on line 283 of schedulers\scheduling_euler_discrete.py

https://github.com/huggingface/diffusers/blob/76c645d3a641c879384afcb43496f0b7db8cc5cb/src/diffusers/schedulers/scheduling_euler_discrete.py#L283

An exception occurs when sigmas is a Tensor object and not a numpy array, when self.config.interpolation_type == "log_linear"

from this setup code directly above

if self.config.interpolation_type == "linear":
    sigmas = np.interp(timesteps, np.arange(0, len(sigmas)), sigmas)
elif self.config.interpolation_type == "log_linear":
    sigmas = torch.linspace(np.log(sigmas[-1]), np.log(sigmas[0]), num_inference_steps + 1).exp()
else:
    raise ValueError(
        f"{self.config.interpolation_type} is not implemented. Please specify interpolation_type to either"
        " 'linear' or 'log_linear'"
    )

Reproduction

import diffusers
import requests
import PIL.Image
import io

pipeline = diffusers.StableDiffusionLatentUpscalePipeline.from_pretrained('stabilityai/sd-x2-latent-upscaler')

url = "https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/sd2-upscale/low_res_cat.png"
response = requests.get(url)
low_res_img = PIL.Image.open(io.BytesIO(response.content)).convert("RGB")
low_res_img = low_res_img.resize((128, 128))
prompt = "a white cat"

upscaled_image = pipeline(prompt=prompt, image=low_res_img).images[0]
upscaled_image.save("upsampled_cat.png")

Logs

Loading pipeline components...: 100%|██████████| 5/5 [00:00<00:00,  5.13it/s]
Traceback (most recent call last):
  File "test.py", line 15, in <module>
    upscaled_image = pipeline(prompt=prompt, image=low_res_img).images[0]
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "...\venv\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "...\venv\Lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_stable_diffusion_latent_upscale.py", line 413, in __call__
    self.scheduler.set_timesteps(num_inference_steps, device=device)
  File "...\venv\Lib\site-packages\diffusers\schedulers\scheduling_euler_discrete.py", line 283, in set_timesteps
    sigmas = torch.from_numpy(sigmas).to(dtype=torch.float32, device=device)
             ^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: expected np.ndarray (got Tensor)

Process finished with exit code 1

System Info

diffusers 0.24.0 torch 2.0.1

Windows

Who can help?

No response

[yiyixuxu]

@Teriks thanks! do you want to open a PR to fix it? just have to change it to np array

[Senume]

Hi @yiyixuxu, Let me look into the issue.

[Senume]

Raised a pull request.

[nileshkokane01]

@yiyixuxu ,

Shouldn't a simple check below suffice ?

sigmas = torch.from_numpy(sigmas).to(dtype=torch.float32, device=device) if not isinstance(sigmas, torch.Tensor) else sigmas

I just did the changes and it is working. I also do not see a PR for this, if so , can I send one?

[Teriks]

@yiyixuxu ,

Shouldn't a simple check below suffice ?

sigmas = torch.from_numpy(sigmas).to(dtype=torch.float32, device=device) if not isinstance(sigmas, torch.Tensor) else sigmas

I just did the changes and it is working. I also do not see a PR for this, if so , can I send one?

This is pretty much what I had edited it to locally to see if it would work, but I ran out of free time :)

[SlZeroth]

same issue

[bamps53]

@yiyixuxu Is this issue still open? It seems to have been fixed by #6056, and I've confirmed that there is no problem with version 0.25.1.

[SahilCarterr]

Hi @sayakpaul .It's my first issues on diffuser Library guide me

[sayakpaul]

Cc: @yiyixuxu

[tolgacangoz]

I recommended @SahilCarterr to study Diffusion Models comprehensively. After studying, it would be more appropriate to examine previously merged good first issue PRs and try to solve unsolved ones, IMHO.