A technical question: In the "sharpness" implementation, why isn't the gaussian degradation applyed on the unconditional branch but conditional branch?

[lawsonxwl]

According to the SAG paper 5.1(Improving Sample Quality of Diffusion Models Using Self-Attention Guidance), the blur guidance is performed on the unconditional model output, in order to guide the sampling process to move toward better quality output (just like CFG).

But in the script modules/patch.py/patched_sampling_function, the implementation seems so strange...

positive_eps = x - positive_x0
negative_eps = x - negative_x0

alpha = 0.001 * sharpness * global_diffusion_progress

positive_eps_degraded = anisotropic.adaptive_anisotropic_filter(x=positive_eps, g=positive_x0)
positive_eps_degraded_weighted = positive_eps_degraded * alpha + positive_eps * (1.0 - alpha)

final_eps = compute_cfg(uncond=negative_eps, cond=positive_eps_degraded_weighted,
                        cfg_scale=cond_scale, t=global_diffusion_progress)

why isn't the gaussian degradation applied on the unconditional branch but conditional branch?

[lllyasviel]

because eps is anti x0 u either make negative x0 bad or make positive eps bad to make positive x0 better

[lawsonxwl]

thanks for the reply. Is it because removing degraded noise from xt means the degradation will be also removed from xt, so x0 will move in the contrary direction of degradation?

btw could you tell me the architecture of the inpaint patch? bc I may want to do some finetune job, is it just a normal controlnet(like controlnet inpaint in sd1.5)? I have printed the model weights, it's so different from a typical LoRA.