Idea: Fine-tunning loop flow

[knoopx]

I've spent a significant amount of time tweaking the fine-tuning process both by tuning hyperparameters and implementing new techniques. In the end most of the code is the same and most of it is glue code that could be described as nodes very much like comfyui does.

The individual nodes that encode images into latents and text embeddings are already there. No sure however iteration could be implemented and if this is something that requires significant code refactoring. Given the flexibility of comfyui and the diverse training strategies and diffusion models available it would be really powerful.

How feasible you think this is? Is this something somebody else thought of?

[ltdrdata]

https://github.com/comfyanonymous/ComfyUI/pull/731

Actually, I have already implemented and tested the functionality to some extent. The problem arises with nodes that receive multiple inputs. In the first loop, it can wait until all inputs are ready before proceeding. However, starting from the second loop, there may already be cached inputs from previous iterations, causing the loop to be entered based on incomplete inputs as soon as new input arrives. Consequently, there is an issue of performing slightly modified duplicate executions due to subsequent incoming inputs.

In summary, there is a synchronization issue with input values in nodes that receive multiple inputs. I am currently contemplating solutions to this problem.

[knoopx]

@ltdrdata great! keeping an eye, will take a chance at implementing the fine-tuning loop when these issues are resolved