cog-sdxl icon indicating copy to clipboard operation
cog-sdxl copied to clipboard

"refine_steps" argument name and docs are misleading

Open iuliaturc opened this issue 9 months ago • 0 comments

When setting refiner=base_image_refiner, the refine_steps argument promises to control how many denoising steps will be performed with the refiner:

Screenshot 2024-05-09 at 11 39 35 AM

However, in practice, this is only true if you additional set prompt_strength=1.0. The reason is that:

  • cog-sdxl sets num_inference_steps=refiner_steps here, and
  • diffusers computes the init_step for refinement based both on num_inference_steps and strength here.

This results in surprising behavior, where you set refiner_steps=N only to see that it ran for fewer steps. I think you could either:

  • Update your documentation to reflect this reality, or
  • Automatically set strength=1 here

iuliaturc avatar May 09 '24 18:05 iuliaturc