axodox-machinelearning icon indicating copy to clipboard operation
axodox-machinelearning copied to clipboard

[Feature Request] Support v-prediction

Open ZDisket opened this issue 4 months ago • 0 comments

Some SD finetunes (and the official Stable Diffusion 2) use v-prediction because they observe that it's better Currently, v-prediction models will export with Olive and run fine with the ONNX diffusers pipeline, but not here. This is "a castle" on one I exported a castle - euler unmod

I tried modifying DpmPlusPlus2MScheduler::ApplyStep by modifying the predictedOriginalSample based off one of the diffusers implementations

        auto predictedOriginalSample = output.BinaryOperation<float>(input, [currentSigma](float a, float b) {
            float sigmaSquared = currentSigma * currentSigma;
            return (a * (-currentSigma / sqrt(sigmaSquared + 1))) + (b / sqrt(sigmaSquared + 1));
            });

~~I could only make the result semi-coherent~~

In DPMPlusPlus2MScheduler, the formula from HF's euler discrete works

        auto predictedOriginalSample = output.BinaryOperation<float>(input, [currentSigma](float model_output, float sample){
            float sigmaSquaredPlusOne = currentSigma * currentSigma + 1;
            return (model_output * (-currentSigma / std::sqrt(sigmaSquaredPlusOne))) + (sample / sigmaSquaredPlusOne);
            });

a castle - dpm fix

If you could finish this ~~(haven't figured out EulerAncestralScheduler)~~ (I have everything here), it would greatly improve compatibility. I uploaded the model in case it's useful.

ZDisket avatar Feb 25 '24 22:02 ZDisket