Skimmed_CFG
Skimmed_CFG copied to clipboard
Intuition behind this node
First of all, thank you for this node! It is awesome. I would love to understand the intuition behind the simplest version of this node and what it is doing under the hood. Please correct me if I'm wrong:
Given the guidance scale, the current latents, the current unconditional noise prediction, and the current conditional noise prediction, this node will:
- Calculate what would be the noise prediction with classifier free guidance using the normal guidance scale
- Figure out where the noise prediction with CFG is overblown by:
- 2.a) Looking at where the signs of the conditional_noise_prediction match the ones from the difference between the conditional and the unconditional noise prediction
- 2.b) Looking at where the signs of the conditional_noise_prediction match the ones from the noise prediction with CFG
- 2.c) (optional) Looking at where the signs of the noise prediction with CFG match the ones from the difference between the noise prediction with CFG and the latents Only when those three conditions coincide, then you determine that the noise pred with CFG is probably overblown
- You calculate a "new" "noise prediction with classifier free guidance", but the scale you use is now the skimming scale
- For the cases where the conditions are satisfied, you replace the conditional noise prediction with the difference between the original conditional noise prediction and the skimmed noise prediction, scaled by the guidance scale
The part I cannot get an intuition for is why you're doing this also for the unconditional noise prediction with the already-processed tensor, rather than with the original unconditional noise prediction.