stable-diffusion-webui-forge
stable-diffusion-webui-forge copied to clipboard
lllyasviel
Fix loras having a weak effect when applied on fp8. - [Comfy Pushed fix for thsi - Can we get in Forge]
Please see this commit that Comfy pushed earlier today that fixes the issue where some Flux LoRA are very weak when using along w/ fp8. It would be great if Forge were similarly updated so there is consistency:
https://github.com/comfyanonymous/ComfyUI/commit/bb222ceddb232aafafa99cd4dec38b3719c29d7d
Hi, we promise that Forge will never secretly add Gaussian noise to your carefully trained LoRAs, without letting users knowing about it. Stick with Forge to have technical correctness and transparency.
Hi, we promise that Forge will never secretly add Gaussian noise to your carefully trained LoRAs, without letting users knowing about it. Stick with Forge to have technical correctness and transparency.
Your answer is over my head at the moment, but going to research. Appreciate your reply and your efforts w/ Forge + Flux!!
BTW, if this should be closed by all means please do so.
This is an issue I also have comparing it to comfy ui. maybe we can have a more technical response on this when you have time.
The code in the link is about rounding exponential values ( Stochastic Rounding Logic ). I didn't check where that function is used but it will introduce inaccuracy and randomness. It will most likely add "noise" to the output as lllyasviel mentioned, and non deterministic noise to be exact.
The code in the link is about rounding exponential values ( Stochastic Rounding Logic ). I didn't check where that function is used but it will introduce inaccuracy and randomness. It will most likely add "noise" to the output as lllyasviel mentioned, and non deterministic noise to be exact.
I'm a layman, but there has to be some loss between a LoRA in FP16 and FP8. It seems that stochastic rounding may just be a better (or worse) way to guess at the values vs. just treating everything the same.
It wouldn't be an annoyance if al LoRA acted similarly to their strength values , but at the moment LoRA trained w/ Ostis' trainer can use the same strength value (1) regardless of FP8/FP16 inference settings. On the other hand, LoRA trained w/ Kohya/SimpleTuner used w/ certain FP8 settings require the strength to be jacked up to 2, 3, or 3+ to see any effect on the image generated by the model.
Think it's kinda a choice on how you want to deal w/ loss incurred when interpreting an FP16 file at FP8.
The code in the link is about rounding exponential values ( Stochastic Rounding Logic ). I didn't check where that function is used but it will introduce inaccuracy and randomness. It will most likely add "noise" to the output as lllyasviel mentioned, and non deterministic noise to be exact.
I'm a layman, but there has to be some loss between a LoRA in FP16 and FP8. It seems that stochastic rounding may just be a better (or worse) way to guess at the values vs. just treating everything the same.
It wouldn't be an annoyance if al LoRA acted similarly to their strength values , but at the moment LoRA trained w/ Ostis' trainer can use the same strength value (1) regardless of FP8/FP16 inference settings. On the other hand, LoRA trained w/ Kohya/SimpleTuner used w/ certain FP8 settings require the strength to be jacked up to 2, 3, or 3+ to see any effect on the image generated by the model.
Think it's kinda a choice on how you want to deal w/ loss incurred when interpreting an FP16 file at FP8.
Is this really confirmed? It does look chaotic. I wonder is most users even know about it. If that is the case we would need to have a big warning on all LoRas page on Civitai for example if the LoRa is trained with one trainer or the other... From what I get, Ostris train on FP8, maybe that is why it works on FP8 normally. Have not tested it on FP16.
Imho ostris lora is way higher quality but i’m comparing with kohya codebase from approx 7 days ago when sd3 scripts were not updated. In any case training process is improved with ostris from time and vram usage perspective. I’m sure kohya will catch up, the app is more complex than the ai-toolkit so it’s not as easy to update I imagine.On 21 Aug 2024, at 16:00, diodiogod @.***> wrote:
The code in the link is about rounding exponential values ( Stochastic Rounding Logic ). I didn't check where that function is used but it will introduce inaccuracy and randomness. It will most likely add "noise" to the output as lllyasviel mentioned, and non deterministic noise to be exact.
I'm a layman, but there has to be some loss between a LoRA in FP16 and FP8. It seems that stochastic rounding may just be a better (or worse) way to guess at the values vs. just treating everything the same. It wouldn't be an annoyance if al LoRA acted similarly to their strength values , but at the moment LoRA trained w/ Ostis' trainer can use the same strength value (1) regardless of FP8/FP16 inference settings. On the other hand, LoRA trained w/ Kohya/SimpleTuner used w/ certain FP8 settings require the strength to be jacked up to 2, 3, or 3+ to see any effect on the image generated by the model. Think it's kinda a choice on how you want to deal w/ loss incurred when interpreting an FP16 file at FP8.
Is this really confirmed? It does look chaotic. I wonder is most users even know about it. If that is the case we should a big warning on all LoRas page on Civitai for example if the LoRa is trained with one trainer or the other...
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>