comfyanonymous
Some requests for LORA training in ComfyUI.
Feature Idea
1: Training with videos for video models. Only images are supported right now.
2: Some optimizers are "schedule free" and some optimizers don't require any hyperparameters, so extra optimizers would be so nice to have access to.
3: There are some methods to improve iteration and convergence speed greatly:
The relatively simple to implement: https://github.com/compvis/tread | https://github.com/vita-epfl/LayerSync | https://github.com/vvvvvjdy/SRA
Slightly harder to implement: https://github.com/Martinser/REG
There are some more mentioned here https://x.com/SwayStar123/status/1994673352754270318
Even just TREAD which seems to be the easiest to implement and is tested for LORA training will GREATLY improve convergence and iteration speed. It would be awesome if some of these get implemented.
@KohakuBlueleaf
Existing Solutions
No response
Other
No response
