stable-diffusion-webui
stable-diffusion-webui copied to clipboard
[Bug]: error when training embedding
Is there an existing issue for this?
- [X] I have searched the existing issues and checked the recent builds/commits
What happened?
Applying cross attention optimization (Doggettx). Training at rate of 0.05 until step 25000 Preparing dataset... 100%|██████████████████████████████████████████████████████████████████████████████████| 10/10 [00:01<00:00, 5.29it/s] 0%| | 0/5891 [00:00<?, ?it/s]Traceback (most recent call last): File "F:\sd\modules\textual_inversion\textual_inversion.py", line 506, in train_embedding scaler.step(optimizer) File "F:\sd\venv\lib\site-packages\torch\cuda\amp\grad_scaler.py", line 336, in step assert len(optimizer_state["found_inf_per_device"]) > 0, "No inf checks were recorded for this optimizer." AssertionError: No inf checks were recorded for this optimizer.
Applying cross attention optimization (Doggettx).
Steps to reproduce the problem
- just train embedding
It happens randomly but when it happens its hard to get it working, and i have no clue why its happening so no idea how to make it go away im on win10 gtx 1080ti
What should have happened?
it should train normally cause it trained seconds ago
Commit where the problem happens
https://github.com/AUTOMATIC1111/stable-diffusion-webui/commits/master
What platforms do you use to access the UI ?
Windows
What browsers do you use to access the UI ?
Google Chrome
Command Line Arguments
none
List of extensions
none
Console logs
none
Additional information
No response
Ok i noticed its related to using particular txt file from template prompts, this should not happen at all, so id call it a bug, i had none.txt selected, it errors out, when i select style.txr it works fine
same error
same
If you are using a Prompt Template txt file, try removing any empty lines. Each line should have a prompt without any blank rows between.
If you are using a Prompt Template txt file, try removing any empty lines. Each line should have a prompt without any blank rows between.
And if there are no empty lines, just a single one and this error still appears ... ? 🤔
Same