diffusion_distiller icon indicating copy to clipboard operation
diffusion_distiller copied to clipboard

Exploding target in distillation

Open HReynaud opened this issue 1 year ago • 0 comments

Hello @Hramchenko ! Thank you for your work on this repo.

I am trying to implement the distillation for v-parametrization and used your code. I am having an issue with this line, because sigma_s can be very small (ex: 1e-2) which causes eps_2 to become very large and thus v_2 too, while it should remain close to the [-1,1] range.

Would you have any insights on that ? Did you derive the equations yourself ?

HReynaud avatar Feb 13 '23 20:02 HReynaud