DarkAlchy

I can tell you a 4090 user (the one I mentioned earlier that you may have skipped over reading) is having the EXACT same issue as I am having so...

There is my friend I was helping with the 4090 and the same results as I was getting. You seriously have an issue with this software, so let's figure out...

> I am using CUDA 11.8.0. Could you please ask your friend to check the CUDA version? As he said above 11.7. He just went to bed but it is...

> I think CUDA 11.7 may work. How much main RAM (not VRAM) is available during script execution? If swapping occurs, the training speed will be greatly reduced. He has...

> Unfortunately, the script implementations are so different that it seems difficult to make a simple comparison. > > Can you try it removing `--use_8bit_adam` option and adding `--cache_latents` option?...

> Removing `--xformers` and adding `--mem_eff_attn` also did not work? Just tried that and it made it only twice as slow as Automatic1111 (down from ~46s to 7.5s).

> Ok thanks! There seems to be a problem with xformers. Unfortunately, I don't have a GTX10xx environment, so I can't test it, but the official xformers wheel seems to...

> It certainly does not explain his issue, but the version of xformers in `README.md` seems to work in my 4090... > > Apparently xformers is very sensitive to the...

@BowenBao Not sure to be honest, but I moved on and away from embeddings into LoRA. I tried the workers Kohya suggest to no avail and gave up on this.

I can't get Oobabooga to train a LoRA for Wizard and it needs more modern code to work with what I am doing. When it thinks that 1.11.x of Pytorch...