Blurry images since today

[lhovav]

Hi,

I'm getting blurry sample images every 250 images,

started today: What I did that I think it's a bug:

1. tried different seed - still blurry
2. tried two different datasets from yesterday (which were perfect) - still blurry,
3. deployed another pod (I'm running RunPod) - still blurry

also, the linkess is meh, compared to yesterday (the .safetensor doesn't produce blurry images on local ComfyUI)

Everything is the same, I'm using the same config.yaml file, the same file structure, just changing the lora's name.

does anyone experience this issue too?

Thanks

[steveant]

I've had the same issue. Besides what you mentioned I also:

• reinstalled locally on windows and on linux with latest (followed README.md process)
• I noticed the samples generated have different levels of fogginess and some are consistently clear. (odd).
• Next I'm going to try to revert to an an older changeset hoping it's an issue in the main repo and not the submodules...

FWIW - the first round of latents that get saved to output//samples before any training are foggy.

[alexpuliatti]

yes here too.

[julien-blanchon]

I also have blurry image but it's mainly on white background which is a know Flux problem

[eepson123tw]

same here, has any suggestion?

[SlZeroth]

same

[bvkimball]

Anyone have a solution or workaround for this?

[fstairu]

The same prob with AI Toolkit on Runpod. Guess it might be the AI Toolkit installation issue, as dataset and .yaml worked just nice previously (not Runpod). Does anyone know if that effects only the sampling, or lora training as well (producing the same blurry images in using .safetensor)?

[xdfin99]

same issue today for me

[OctaM]

I have the same issue today. I trained a model yesterday and the images were just fine. Trained another one today and the output is blurry