rgthree-comfy icon indicating copy to clipboard operation
rgthree-comfy copied to clipboard

[ food for thought ] KSampler user profiles ( for each checkpoint type )

Open JorgeR81 opened this issue 1 year ago • 12 comments

This is not a specific request, just something to think about, that could lead to a nice feature, if it's not too hard to implement.

We have SDXL turbo models, LCM models, and now SDXL lightning models, which need different KSampler settings. Also, Stable Cascade, seems to work better with other settings, based on the defaults workflows provided.

So, when we change a checkpoint in the loader, we may also need to change lots of other parameters like: cfg, steps, sampler, scheduler, etc... It's hard to remember all of these values.

Sure, we can create a workflow for each model type. But when I load a new workflow, I need to change the prompts, and I also may need to add other nodes that I was using at the moment. So It's just faster to manually copy the KSampler settings to my current workflow.

And in addition to that, we have community merges of these models, which will work better with slightly different settings.

So, in practice, when I change the checkpoint, I need to find a nice image I created with that checkpoint ( which is not that easy if you don't have the images super organized ). Then, open the workflow in another browser tab, and manually copy the settings to my current workflow.

That's got to be a better way to do that. Right ?

Maybe a KSampler node with another setting for [ profiles ] ?

Or maybe something accessible just via right click, that works on the native KSampler node ? Also with options to "add current settings as profile" and "delete profile".

I think this would work better, if the available profiles can be selected for each KSampler node, instead of linking them to the checkpoint ( loader ). This gives more flexibility.  For instance, we can have profiles for anime or realistic images, independently from the checkpoint we use. Or a profile that works better on a KSampler that's used as an image refiner, in the workflow. Also, in Stable Cascade, we need different KSamplers settings in Stage B and Stage C, so we could have 2 profiles for that.

JorgeR81 avatar Feb 24 '24 12:02 JorgeR81

On second thought, it may not be worth it to create a whole new management system, just for this particular use case.

We already have something similar to manage templates.

And some of these KSampler use cases, also require extra nodes, in addition to special settings ( like SDXL lightning and Stable Cascade ) so this may be more convenient.

templates1 templates2

JorgeR81 avatar Feb 27 '24 13:02 JorgeR81

You actually don't need to use separate samplers and such for each type of model. Cfg and Steps sure, but there's still no "right answer" for any specific model. If you have certain ideal defaults you like, it probably makes sense to create a workflow that can look at the name of the checkpoint, and make decisions based on that checkpoint.

I submitted this PR to get the checkpoint name sent from the loader, just so I could make decisions based on that name. https://github.com/comfyanonymous/ComfyUI/pull/2569 (technically that means I am forced to use a fork of ComfyUI until it's merged).

You can use "Text Contains" to get a substring (from https://github.com/WASasquatch/was-node-suite-comfyui). You can use the "Logic OR" from the same node suite as well. I just submitted a pull request to do a find (supports substring or regex) to WAS: https://github.com/WASasquatch/was-node-suite-comfyui/pull/366.

ghostsquad avatar Mar 16 '24 18:03 ghostsquad

That sounds very interesting ! Once this is all merged, maybe you could share an example workflow.

JorgeR81 avatar Mar 16 '24 18:03 JorgeR81

The settings for each use case / checkpoint, can be "stored" in multiple [ KSampler Config (rgthree) ] nodes, in the workflow. And then switched, via [ Context Big (rgthree) ] nodes

It even works with [ cg-use-everywhere ] nodes !

But now it needs to be toggled manually.

context

JorgeR81 avatar Mar 16 '24 20:03 JorgeR81

Using the [ Fast Groups Muter ], we can go above the 5 input switch limit. Although I don't think this could be controlled based on the checkpoint name. 

gmut3

JorgeR81 avatar Mar 16 '24 21:03 JorgeR81

You can go above the 5 input switch limit by just hooking up multiple switches together.

ghostsquad avatar Mar 16 '24 21:03 ghostsquad

It works ! Thanks !

ctx6

JorgeR81 avatar Mar 16 '24 22:03 JorgeR81

This is a little cumbersome to set up. So here is the workflow.

config_6_ue

And a version without [ cg-use-everywhere ].

config_6

JorgeR81 avatar Mar 16 '24 22:03 JorgeR81

A huge improvement to ComfyUI would be to put outputs circles next to the widgets. That would save so much space!

That should also be a default. Like if the node has a widget, there's an output for that value, but you can also output calculated values.

ghostsquad avatar Mar 17 '24 00:03 ghostsquad

You meant outputs for the string names, or the regular outputs ?

For the regular outputs, it would be a little confusing, if the widget is related to multiple outputs, like the checkpoint loader.

But these alternatives could be node properties. You could use the node's right click menu, to show/enable the name outputs for the widgets or to place the regular outputs in front of the widgets, to get a more compact layout, when it makes sense, for that node.

JorgeR81 avatar Mar 17 '24 01:03 JorgeR81

An even better solution would be for the regular outputs to work both ways.

They could give the name output, if you just connect them to the right kind of input.

ckpt

JorgeR81 avatar Mar 17 '24 02:03 JorgeR81

Just found this loader from the WAS suite !

Isn't this what you need ?

was

JorgeR81 avatar Mar 17 '24 02:03 JorgeR81