deGENERATIVE-SQUAD

fused_backward_pass in prodigy-plus-schedule-free [SOLVED VIA EXTERNAL SOLUTION ✅]

7

Hi, Kohya. I know you hardcoded fused_backward_pass to Adafactor, but prodigy-plus-schedule-free https://github.com/LoganBooker/prodigy-plus-schedule-free has that feature inside already, but we cant use it. That is, in fact, we can apply the...

We have two branches of new, especially precise and “clean” VAEs for SDXL called EQ VAE (one from https://huggingface.co/KBlueLeaf/EQ-SDXL-VAE ). Test trainings show a visually noticeable improvement when using these...

⛔[SD3 branch] Huber+SNR RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

13

Pytorch 2.7.0, CUDA 12.8 What: RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! when using Huber + SNR Where:...

⛔[SD3 branch] NaN when using full FP16 with custom optimizers

3

Problem: When using fp16 and full_fp16 in combination with custom optimizers from the https://github.com/kozistr/pytorch_optimizer library, NaNs appear on the second training step. This issue also occurs when the VAE is...

Sparse learning gradient filtering feature request 🟡

1

The main issue: sd_scripts lacks a custom gradient filtering function with masking. My attempt to implement a module with custom filters via an additional argument was unsuccessful due to my...