[Feature Request]: Why is the SDXL running so slow?

[maxin9966]

Is there an existing issue for this?

• [x] I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

I have noticed that running sdxl in the WebUI is quite slow. I am running the XL-Lightning model, which typically takes 2.5-2.8 seconds for a 1024x1024 resolution, 6-step; however, using the following code reduces the time to around 1.2-1.3 seconds.

code: pipe = StableDiffusionXLPipeline.from_single_file(ckpt_name, torch_dtype=torch.float16, variant="fp16").to("cuda") # DPM++ SDE pipe.scheduler = DPMSolverSinglestepScheduler.from_config(pipe.scheduler.config, use_karras_sigmas="yes", timestep_spacing="trailing")

    image = pipe(prompt=prompt,
                 num_inference_steps=6,
                 guidance_scale=0,
                output_type = "pil",
                width = width,
                height = height,
                cfg_scale = 2.0,
                # ,eta=eta
                 ).images[0]

Proposed workflow

1. Go to ....
2. Press ....
3. ...

Additional information

No response