mliand
mliand
You can show me your loss training
> Lumina 目前正在与 SD3-VAE 集成。该模型预计将在几周后发布。 As far as I know, the basic license of sd3 restricts the use of any components of sd3 for other models, and stability will be...
t5xxl_dtype=bf16
same problem, some optimizers and schedulers seem to have problems when calculating gradients
I think you should try to examine the data set
I've trained on a 16k dataset, but it seems that mixed precision training is problematic, and my average loss is in the 0.09 range
> > 我已经在 16k 数据集上进行了训练，但混合精度训练似乎存在问题，我的平均损失在 0.09 范围内 > > 如果您没有遇到 NaN，kohya 也不应该遇到它。您能告诉我们您使用的学习率和重复率吗？也许 kohya 只是使用了太高的 lr？在今天之前，混合精度是有问题的。Bf16 将导致在今天之前启用 --full_bf16。另外，您的训练的起始损失是多少？  mixed_precision = "bf16" full_bf16 = false weighting_scheme = "logit_normal" sdpa = true...