ComfyUI icon indicating copy to clipboard operation
ComfyUI copied to clipboard

Flux output is blurry compared to webui-forge

Open Foul-Tarnished opened this issue 1 year ago • 13 comments

I use your workflow (https://comfyanonymous.github.io/ComfyUI_examples/flux/) but with the GGUF extension (https://github.com/city96/ComfyUI-GGUF) Q6_K and t5xxl fp8

I tried to replicate a gen on forge to compare, and it seems it's not just a model/prompt issue: (Check in fullscreen to better see)

ComfyUI

SDWebUI-Forge

It's strange because even though half the generations are blurry on both (model/prompt issue), comfy does appear blurrier, always*. It might be even more visible on sharper image (better prompt)

Other

  • Fresh ComfyUI standalone install, then upgraded to Pytorch 12.4, on RTX 4080
  • I tried everything I could think of to replicate (CPU noise on forge, same sampling method and schedule type, same sampling steps)
  • Could be related to the GGUF extension, as I didn't check on FP8/FP16

Foul-Tarnished avatar Aug 21 '24 01:08 Foul-Tarnished

I commented on your post in the GGUF repo: https://github.com/city96/ComfyUI-GGUF/issues/54#issuecomment-2300129278

RandomGitUser321 avatar Aug 21 '24 01:08 RandomGitUser321

Have you tried other resolutions? I tested some SDXL recommended resolutions without blurry results.

deepfree2023 avatar Aug 21 '24 14:08 deepfree2023

Tried some higher resolutions, blurry occured. Need be confirmed.

deepfree2023 avatar Aug 21 '24 14:08 deepfree2023

When I reproduced using the CPU seed, I obtained completely identical results.

comfyui-workflow

forge

ComfyUI: comfyui

Forge: forge

Both were reproduced in the latest version.

Please share more detailed reproduction conditions.

ltdrdata avatar Aug 21 '24 16:08 ltdrdata

Replacing RandomNoise node by RandomNoise (Inspire) node in Foul-Tarnished's comfyUI workflow, and set noise_mode to cpu, the result was just the same as using RandomNoise node.

So It's already cpu seed in the comfyUI workflow.

deepfree2023 avatar Aug 21 '24 22:08 deepfree2023

This workflow can be loaded to replicate the blurry ComfyUI image: workflow

It would be interesting to find out how can forge produce a sharper image without much detail difference to the blurry one?

I've heard some people reporting higher probability of producing blurry images using dev model, and encounted this kind of problem myself.

deepfree2023 avatar Aug 22 '24 06:08 deepfree2023

This workflow can be loaded to replicate the blurry ComfyUI image: workflow

It would be interesting to find out how can forge produce a sharper image without much detail difference to the blurry one?

I've heard some people reporting higher probability of producing blurry images using dev model, and encounted this kind of problem myself.

Although I haven't tested it, disabling the ModelSamplingFlux node should result in a configuration identical to Forge. And is it really true that the beta schedule was applied in Forge?

ltdrdata avatar Aug 22 '24 06:08 ltdrdata

This workflow can be loaded to replicate the blurry ComfyUI image: workflow It would be interesting to find out how can forge produce a sharper image without much detail difference to the blurry one? I've heard some people reporting higher probability of producing blurry images using dev model, and encounted this kind of problem myself.

Although I haven't tested it, disabling the ModelSamplingFlux node should result in a configuration identical to Forge. And is it really true that the beta schedule was applied in Forge?

Disabled ModelSamplingFlux node , by using normal, simple or sgm_uniform, the result would be sharper, although not the same as Tarnished's Forge image, and still somewhat blurry.

ComfyUI_00542_

deepfree2023 avatar Aug 22 '24 07:08 deepfree2023

This workflow can be loaded to replicate the blurry ComfyUI image: workflow It would be interesting to find out how can forge produce a sharper image without much detail difference to the blurry one? I've heard some people reporting higher probability of producing blurry images using dev model, and encounted this kind of problem myself.

Although I haven't tested it, disabling the ModelSamplingFlux node should result in a configuration identical to Forge. And is it really true that the beta schedule was applied in Forge?

Disabled ModelSamplingFlux node , by using normal, simple or sgm_uniform, the result would be sharper, although not the same as Tarnished's Forge image, and still somewhat blurry.

ComfyUI_00542_

In the beta schedule, both resulted in blurred images, while in the simple schedule, completely identical images were obtained.

When applying the beta schedule, I'm not sure what additional steps Forge takes internally, but there were subtle differences in detail. While ComfyUI produced a slightly more blurred image, Forge's output was also blurred.

It's not that the beta schedule has a tendency to create blurred images. According to my tests, FLUX simply has a tendency to generate blurred images with certain prompts. And when the results are not exactly the same, it just leads to differently blurred images.

ltdrdata avatar Aug 22 '24 15:08 ltdrdata

Made a few more tests in ComfyUI(based on the previous workflow):

  1. Dev models, at 976 x 1328, the blurry image ratio is higher(about 3/8-4/8 in my tests), using different scheduler may increase or decrease the blurry level, but can not make the image really sharp, ModelSamplingFlux does not affect the blurry level obviously. Adjust pompt seems cannot solve the blurry problem(need more tests).
  2. Dev models, at 768 x 1280, the blurry image ratio is lower (about 1/8 in my tests).
  3. Schnell, at 976 x 1328 or 768 x 1280, the blurry image ratio is 0.

(need much more tests to reach any serious conclusion)

deepfree2023 avatar Aug 22 '24 15:08 deepfree2023

"FLUX simply has a tendency to generate blurred images with certain prompts." This could be the case. It might be that to get better image quality, there's a tendency to use a shallow depth of field, which makes it easier to go out of focus?

deepfree2023 avatar Aug 22 '24 15:08 deepfree2023

Tested some fine-tuned model, claiming solved blurry problem, and yes, very sharp images without blurry up to now. But they are still in the early stage, and have other problems. This kind of fine-tuning models or loras might be the solution to the blurry image problem.

@Foul-Tarnished Could you take more tests to confirm if Forge has lower blurry image rate with your configuration?

deepfree2023 avatar Aug 22 '24 18:08 deepfree2023

I use your workflow (https://comfyanonymous.github.io/ComfyUI_examples/flux/) but with the GGUF extension (https://github.com/city96/ComfyUI-GGUF) Q6_K and t5xxl fp8

I tried to replicate a gen on forge to compare, and it seems it's not just a model/prompt issue: (Check in fullscreen to better see)

ComfyUI

SDWebUI-Forge

It's strange because even though half the generations are blurry on both (model/prompt issue), comfy does appear blurrier, always*. It might be even more visible on sharper image (better prompt)

Other

  • Fresh ComfyUI standalone install, then upgraded to Pytorch 12.4, on RTX 4080
  • I tried everything I could think of to replicate (CPU noise on forge, same sampling method and schedule type, same sampling steps)
  • Could be related to the GGUF extension, as I didn't check on FP8/FP16

choose sgm_uniform or beta of ksampler schduler instead of simple,btw,diffusers has FlowMatchEulerDiscreteScheduler which i think flow match will be another option

frankchieng avatar Oct 09 '24 09:10 frankchieng

I still get this, what's the fix?

shaun-ba avatar Oct 23 '24 16:10 shaun-ba

I still get this, what's the fix?

Have you tried antiblur lora or something like that?

deepfree2023 avatar Oct 26 '24 11:10 deepfree2023