stable-diffusion-webui

stable-diffusion-webui copied to clipboard

Added batched learning to textual inversion tab. On large vram cards, batch size larger than 1 may help speed up the training simply cache entries until batch size number of entries collected and go training for one step

As issue https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/1862 suggest

I was already working on this for both TI and hypernets; my version is not merged it. Also looking at code I think you're increasing step by batch count every time, which is not how it's supposed to work as far as I know.

As i tried this batched implementation, it seems large batch size (e.g. 8) generally prohibits learning textual inversion. either with original learning rate(0.005, working when batch size = 1) or scaled by batch size(0.005 * batch size) as mentioned in the original paper. batch size down to 2 is fine with decent acceleration.

I was already working on this for both TI and hypernets; my version is not merged it. Also looking at code I think you're increasing step by batch count every time, which is not how it's supposed to work as far as I know.

embedding.step seems nothing to do with training procedure and just a counter?