ostris
Why the samples from the training are far better than the inference in Comfy?
Hi guys. I trained some Loras for Chroma in AI-Toolkit on a 4090 but I'm having a problem, the samples from the training script look way better than the actual inference in Comfy, does anyone have the same problem or know how to solve this? Could it be a problem with the Lora loaders in Comfy?
The toolkit uses diffusers (e.g., the flowmatcheulerdiscrete sampler) that aren’t available natively in ComfyUI. There is a ComfyUI diffusers node package, but it had broken dependencies for me
Yes, I've encountered the same issue. When training the kontext model, the inference results during training are far better than those in ComfyUI. ComfyUI even completely fails to reproduce the LoRA effects
It could be a million reasons depending on model and workflow. Make sure you are not getting key errors in comfy in the console when loading the lora. Also note, if you are running models quantized or in a low precision like 8bit, applying a lora to an quantized model makes a lot of lora weights get completely rounded away when applied to the model. This is a universal issue due to how quantization works.
It could be a million reasons depending on model and workflow. Make sure you are not getting key errors in comfy in the console when loading the lora. Also note, if you are running models quantized or in a low precision like 8bit, applying a lora to an quantized model makes a lot of lora weights get completely rounded away when applied to the model. This is a universal issue due to how quantization works. I trained a LoRA that removes characters while retaining clothing, which works perfectly fine, proving my approach is correct. However, another LoRA I trained for multi-angle regeneration performs very poorly, making it extremely difficult to turn the character to the left. The flux-kontext model by default is already very challenging when it comes to turning characters leftward.
Yes, I encountered exactly the same problem. I tried to use CustomScheduler node in ComfyUI to use the same sigmas as in ai-toolkit (which I figured out by adding some print statements in the code) so I can use the same scheduler. This did make sample images better, but still not as good as those samples by ai-toolkit. I guess this is because “Euler” sampler in ComfyUI is different from that used in FlowMatchEulerDiscreteScheduler in diffusers. If I am right, I wonder if there is any custom sampler implementation for ComfyUI
@qiujunhan hello，Have you solved this problem? I noticed that for relatively simple tasks, such as inputting a single image and using instructions to tell it to remove the main subject, it performs very well. The samples during training and the inference in ComfyUI are both good. However, when I trained another LoRA, where the input became about 2 images stitched together left and right, and the output was a vertically proportioned image, the sample effect during the training process basically met expectations, but the effect during inference in ComfyUI was much worse. The proportions of the characters also became unreasonable, and the characters looked like they were squashed. If you know how to solve it, thank you very much.
In my experience, the mismatch of model precision, such as training in fp16 but inferencing in fp8, or vice versa, or training in fp8 but Comfy setting it to fp16, often significantly degrades the output quality. This can happen to text encoders, UNETs, and VAEs. It might be necessary to check each one individually. However, the sampler shouldn’t be the culprit.
Appears there's a similar problem for qwen-image-edit-2509; lora had absolutely no effect in a fp8 setup, but was fine in the training samples. (I've not choice but to use fp8 locally)
Some very minor success increasing the lora strength, but was burning out long before it ever reached the expected effect. I might re-attempt training with vanilla qwen-image-edit since it's semi-forwards compatible, or worst case setup comfyui with higher compute to use fp16 and confirm directly if it's quantization that is the complete problem.
Just a FYI from my experience training a Wan2.2 LORA. Like mentioned, the samples in AI Toolkit were great, but in Comfyui were not even close. Came down to using Kijai's WanVideoWrapper workflow. Using Kijai's workflow I couldn't get the LORA to work at all and all I was doing was just generating 1 frame to test. Moved to native Comfyui workflow and the LORA works perfect so not sure what's going on with Kijai's wrapper. Seems to be his sampler node that is causing the issue I believe.
