sd-scripts Implement exponential moving averages (EMA)

Implement exponential moving averages (EMA)

Open ddpasa opened this issue 1 year ago • 3 comments

using EMA during training often leads to better results, and is a standard technique these days in training models.

May 07 '24 07:05 ddpasa

--enable_ema

May 07 '24 12:05 ThereforeGames

--enable_ema

I can't find this option in the current codebase. Was it taken out?

May 07 '24 15:05 ddpasa

My apologies, I've been using the option for so long that I forgot it wasn't a standard part of the codebase. 😅

It's from this PR:

https://github.com/kohya-ss/sd-scripts/pull/893

There is also a newer (better?) implementation here, but that one appears to have been canceled.

May 08 '24 07:05 ThereforeGames