tiled_upscaling outputs terrible artifacts, issues with launching it right

[Pirog17000]

Describe the bug

I'm preparing pipeline for upscale this way:

 pipe = StableDiffusionUpscalePipeline.from_pretrained("stabilityai/stable-diffusion-2-1", 
                                                       use_auth_token="12345_token_4321",
                                                       local_files_only=False,
                                                       revision='fp16',
                                                       torch_dtype=torch.float16,
                                                       custom_pipeline="tiled_upscaling",
                                                       low_cpu_mem_usage=True,
                                                       low_res_scheduler=PNDMScheduler,
                                                       )

This always leads to error:

\lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_stable_diffusion_upscale.py", line 427, in __cal
l__
    image = self.low_res_scheduler.add_noise(image, noise, noise_level)
TypeError: PNDMScheduler.add_noise() missing 1 required positional argument: 'timesteps'

how do I properly launch this custom pipeline? I'm missing something?

Reproduction

Try to launch it, that's the reproduction of the bug. Pipeline doesn't launch at all :/

Logs

No response

System Info

• diffusers version: 0.10.2
• Platform: Windows-10-10.0.19045-SP0
• Python version: 3.10.5
• PyTorch version (GPU?): 1.13.0+cu116 (True)
• Huggingface_hub version: 0.11.0
• Transformers version: 4.25.1

link to tiled upscaling pipeline

[Pirog17000]

def upscale_callback(obj):
    print(f"progress: {obj['progress']:.4f}")
    obj['image'].save("diffusers_library_progress.jpg")

. . . . 

        pipe = StableDiffusionUpscalePipeline.from_pretrained("stabilityai/stable-diffusion-x4-upscaler",
                                                           revision='fp16',
                                                           custom_pipeline="tiled_upscaling",
                                                           torch_dtype=torch.float16,
                                                           low_cpu_mem_usage=True,
                                                           #low_res_scheduler=DDIMScheduler,
                                                           )

        image = pipe(prompt=current_prompt,
                     image=image_to_upscale,
                     num_inference_steps=parameters['steps'],
                     negative_prompt=current_negative,
                     guidance_scale=parameters['scale'],
                     noise_level=parameters['repaint_amount'], callback=upscale_callback
                     )

        image.save(upscale_path)

This actually works, but the output is completely garbage from the beginning. And I believe the issue is in the scheduler. What's the way of using a different scheduler with it? The way I've tried to feed DDIM didn't work, it was saying: should be <class 'diffusers.schedulers.scheduling_utils.SchedulerMixin'> no idea 🤔

[Pirog17000]

Image output example:

[Pirog17000]

pipe = StableDiffusionUpscalePipeline.from_pretrained("stabilityai/stable-diffusion-x4-upscaler",
                                                           revision='fp16',
                                                           custom_pipeline="tiled_upscaling",
                                                           torch_dtype=torch.float16,
                                                           low_cpu_mem_usage=True,
                                                           low_res_scheduler=DDIMScheduler(),
                                                           )

okay, now it doesn't bugging, but still output is really bad.

[Pirog17000]

@peterwilli

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[peterwilli]

Hey everyone!! I have awoken from my slumber. Firstly, I'm surprised you got it to run it at all, because I was asking for help getting it to run, and instead they merged my PR. Anyway, it's good to see that it runs!

As for the bad quality. The best params I found was to set noise_level to 50. It's a little on the high end, but it worked for us.

Lmk if you need more help!

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.