ComfyUI_UltimateSDUpscale icon indicating copy to clipboard operation
ComfyUI_UltimateSDUpscale copied to clipboard

sdxl base model and refiner support

Open osfa opened this issue 1 year ago • 10 comments

does this support running a base model and a refiner somehow? seems to work in the automatic extension.

osfa avatar Sep 27 '23 13:09 osfa

What are you looking for it to do? Have you tried using the base and refiner?

ssitu avatar Sep 27 '23 13:09 ssitu

use the base and refiner in conjunction (first some steps base model, then some steps refiner) and pipe them into the ultimate upscaler. from what I can see and from all different examples only one or the other is used as the ultimate upscale node only takes one model as input. am I missing something?

osfa avatar Sep 27 '23 15:09 osfa

I can definitely add that, but I'm not sure how it should look like. Is there a custom node that does what you're looking for that I could look at?

ssitu avatar Sep 27 '23 15:09 ssitu

https://github.com/SytanSD/Sytan-SDXL-ComfyUI something like this? so it receives a latent instead of a model maybe?

osfa avatar Sep 27 '23 16:09 osfa

Well you do need the model to add detail to the image.

ssitu avatar Sep 28 '23 20:09 ssitu

Sorry don't mean to but in butt I have a related question. @ssitu if I plugin the base sdxl model to Ultimate Upscaler Model input I get the following error. Any ideas?

Error occurred when executing UltimateSDUpscale:

mat1 and mat2 shapes cannot be multiplied (308x1280 and 2048x640)

I can use the refiner as the model input however the image ends up a bit garbled.

Thank you!

Acephalia avatar Oct 07 '23 07:10 Acephalia

@Acephalia can you share the workflow with the problem? And are you using a low denoise for the refiner?

ssitu avatar Oct 07 '23 15:10 ssitu

ok yeah so not instead of, but the ability to pass a model chain sampler chain into the module. would this be super complicated?

re: the error, don't think it's related. sounds like a mismatch of model resolutions/versions, probably running something in 512 on 768 stabdiff 2 models or something? controlnet 1 on a sdxl model?

osfa avatar Oct 08 '23 11:10 osfa

ok yeah so not instead of, but the ability to pass a model chain sampler chain into the module. would this be super complicated?

re: the error, don't think it's related. sounds like a mismatch of model resolutions/versions, probably running something in 512 on 768 stabdiff 2 models or something? controlnet 1 on a sdxl model?

"Related question" I.e related to SDXL Base and Refiner model support. What I was trying to do was very similar to what you had mentioned and I didn't want to open a new thread unless it was necessary. I've just cut back to a basic workflow and attempting to plug in USDU and still getting the error on the refiner model works fine with base. I could very well be missing something obvious or maybe I just can't use the refiner as the input for the model?

@ssitu I do have the workflow for you. Should I open a new thread? Definitely don't intend to step on any toes.

Acephalia avatar Oct 09 '23 08:10 Acephalia

@Acephalia It works with the base but not with the refiner? I can't run SDXL but I can take a look at the workflow to see if I can notice any issues. No need for a new thread, this is pretty related.

ssitu avatar Oct 10 '23 20:10 ssitu