Doesn't make sense to separate pre-processor from model?

[ghostsquad]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

I could be wrong about this, please let me know if I am. It doesn't make sense have separate pre-processor drop down and a model dropdown. They are more or less 1-1 relationships.

Steps to reproduce the problem

With 2 drop downs, it makes it impossible to use the X/Y/Z script, as I would have to put preprocessor in X and model in Y, but that doesn't make sense. That creates permutations that won't work.

What should have happened?

I'd like to be able to combine the pre-processor and model as "1 change" so I can generate images using openpose, canny, scribble, depth.

Commit where the problem happens

webui: 27e319dc ( controlnet: 274dd5df (Tue Mar 14 00:04:36 2023)

What browsers do you use to access the UI ?

Mozilla Firefox

Command Line Arguments

No

Console logs

N/a

Additional information

No response

[2blackbar]

i agree , you mostly get funky results from mixing them up, it would be better if you would just choose model and preprocessor would be loaded , its gonan be hard to memorize what preprocessor is for what weights, maybe some hints in webui itself would help with this ? Currenty the solution is to rename weights and include preprocessor name in them, like clipseg for style

If this gets implemented it would be nice to have "dont load preprocessor " tick box if you dont need it

[ljleb]

Maybe add an option "load related preprocessor when changing model" in settings? We could check it by default and if you want the ui to stop selecting preprocessor automatically you just uncheck.

We could alternatively add an "auto" option to preprocessor list that picks the right module depending on model type. Put it right under "none" for convenience. Advantage of this approach is that you can detect the preprocessor from API as well.

[ghostsquad]

Ooo... here's an idea (may require a little bit more coding). Simply adding a new field called processor+model and that field is automatically populated with the processor you select and the model you select, or vice versa. It's a read-only field, that would be most useful with the X/Y/Z plot script.

As mentioned, there are definitely other features here, where maybe preprocessor should be filtered based on the selected model, to make it easier on the user to select the right one. It would have the additional advantage of defaulting to the basic preprocessor for that model.

If you combine that with a checkbox for enable preprocessing (which, when unchecked, would disable the preprocessor dropdown menu), then, that makes it more explicit about what's happening.

[marcsyp]

I agree with the basic premise of this issue, however, a few things to consider as you think about a solution:

• Sometimes you want a model on but no preprocessor, because you are feeding images that have already been preprocessed or looking for experimental results
• Some models have more than one preprocessor that is appropriate -- most obvious is depth with depth and depth_leres options
• This is actually a general problem with X/Y/Z, not necessarily with the options provided for ControlNet. Sometimes you want to iterate on parameters that have a relationship to each other and don't want all the combinations, but just pairs.

To this point: This may actually be solved better by modifying the XYZ script to allow multiple fields for each dimension that follow a "matching pattern" rather than a "combinatorial pattern". Imagine in X, for instance, you could select model and type in "control_sd15_depth, control_sd15_depth, control_sd15_canny" (or whatever), and then you could click a "+" button that would add a line where you select preprocessor and type "depth, depth_leres, canny" It would match the items in the list by index (with some list length resolution logic for mismatched lists) and that would constitute your X dimension, which then gets permutated against whatever you choose for Y and Z.

This solution also would allow for general purpose solutions for testing known configurations of general parameters. Lets say I know that I have CFG + Steps combinations that work particularly well together for a certain prompt, and I want to vary some other parameters (like ControlNet variables) I could create CFG/Steps pairs of "6.5, 7.5, 9.5" and "23, 36, 39" and then use those 3 pairs to test ControlNet Guidance Stop and Strength on Y and Z.

[ghostsquad]

To this point: This may actually be solved better by modifying the XYZ script to allow multiple fields for each dimension that follow a "matching pattern" rather than a "combinatorial pattern". Imagine in X, for instance, you could select model and type in "control_sd15_depth, control_sd15_depth, control_sd15_canny" (or whatever), and then you could click a "+" button that would add a line where you select preprocessor and type "depth, depth_leres, canny" It would match the items in the list by index (with some list length resolution logic for mismatched lists) and that would constitute your X dimension, which then gets permutated against whatever you choose for Y and Z.

I suppose this is already solved through batching. You can create multiple text files, each with all the various fields that you want filled in, and it will essentially iterate through that batch of files.

[jonesi13]

Yes, having an automated way for loading the preprocessor would be extremely helpful. At the moment it's just easier to have multiple CN tabs opened with different models selected and just switch between them, instead of changing the model in one. Having to select two dropdowns every time just takes too much time if you constantly switch.