ComfyUI icon indicating copy to clipboard operation
ComfyUI copied to clipboard

Add multi-IO reroute nodes

Open rhdunn opened this issue 1 year ago • 5 comments

Use Case

Make managing muti-node buses easier, for example:

  1. image latent and VAE -- a 2-channel reroute bus;
  2. positive and negative prompt -- a 2-channel (or 4-channel with L and G variants) reroute bus;
  3. model, positive encoded prompt, negative encoded prompt -- a 3-channel reroute bus;
  4. configuration parameters -- a multi-channel reroute bus depending on the number of parameters, e.g. 2 for a width/height pair.

Required Nodes

  • Reroute2 -- a 2 input/2 output reroute node, e.g. (latent, via) for VAE Encode
  • Reroute3 -- a 3 input/3 output reroute node, e.g. (model, clip, vae) from Load Checkpoint
  • Reroute4 -- a 4 input/4 output reroute node, e.g. (model, clip, positive, negative) for a text encode pipeline
  • Reroute5 -- a 5 input/5 output reroute node, e.g. (model, clip, positive_{l,g}, negative_{l,g}) for an SD XL text encode pipeline

Optional Nodes

  • Reroute6 -- a 6 input/6 output reroute node
  • etc.

Existing Plugin: ImpackPack (Pipes)

This provides an alternative solution to this problem by creating node-specific pipes:

  • basic pipe -- a 5-channel pipe for model data
  • detailer pipe -- a 9-channel pipe for detailer nodes
  • detailer sd xl pipe -- a 13-channel pipe for SD XL detailer nodes

The pipe approach is complex and limited in that it only supports a limited number of pipes, requires from pipes to get the grouped nodes, and requires pipe-specific nodes to handle the grouped inputs.

Reroute Pipe Fan In / Fan Out

A more flexible multi-IO pipe system (i.e. a tuple data line) is possible on top of this. This may be a separate issue.

The basic idea to support this would be to have FanInN and FanOutN variants of RouteN where:

  • FanInN would have N inputs and 1 output;
  • FanOutN would have 1 input and N outputs.

Note: I prefer having the RerouteN variants as it makes it easier to do pull specific outputs to parts of the pipeline (e.g. if it just requires the model) and continue the bus to the next part of the pipeline.

rhdunn avatar Jun 24 '24 16:06 rhdunn

I've had a look at implementing this as a custom Python node and ran into issues with allowing any types on inputs that forward the actual type to the corresponding output type.

This makes it impossible to implement this functionality as an extension in a way that is usable in different contexts. (This is one of the reasons ImpactPack has fixed type inputs.)

Looking at the code, the RerouteNode class in the JavaScript is a special node type with graph/type checking. It is doing the forwarding in the code. The RerouteNNode classes could be implemented by either generalizing the RerouteNode class, or as a class that contains N RerouteNode instances.

rhdunn avatar Jun 24 '24 18:06 rhdunn

Yes that would definitely need to be implemented in JavaScript rather than Python for the current codebase.

It would potentially make more sense as an extension than in the core - but could you explain why you want this / what benefit it would bring to workflow usage?

mcmonkey4eva avatar Jun 24 '24 20:06 mcmonkey4eva

The things in the use cases I listed above are things I'm considering on grouping.

I've been organizing the through data in workflows into data buses that move through the workflow. For example:

sdxl-workflow

I'm looking at better ways of grouping these together. -- I'm not keen on the ImpactPack solution of a single pipe as you cannot easily add to the pipe or pull specific data off the pipe.

As a workaround I'm creating data-specific custom bus nodes for my workflow. For example: latent-image-bus

Note: one potential issue is that the RerouteNode has an explicit check for the "Reroute" node when checking the connections, so it would be tricky implementing a generic solution as a plugin.

rhdunn avatar Jun 24 '24 22:06 rhdunn

For reference, I've created https://github.com/rhdunn/comfyui-bus-plugin which has a collection of these different node types that I'm finding useful in my own workflows. One advantage I've noticed with this approach is the ability to use the nodes as a grouped set of primitive nodes -- i.e. as an input group.

rhdunn avatar Jun 25 '24 17:06 rhdunn

Take a look at hits https://github.com/Trung0246/ComfyUI-0246

ltdrdata avatar Jun 26 '24 11:06 ltdrdata