[Bug]: Newly created embeddings considered malicious

[msoucy]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

Newly created embedding (or any other embedding for that matter) fails to load

Freshly created embedding is attached

Steps to reproduce the problem

1. Go to Train > Create Embedding
2. Create an embedding with any name and start data
3. Check terminal window

What should have happened?

Embedding should be created and load (albeit empty)

Commit where the problem happens

737eb28faca8be2bb996ee0930ec77d1f7ebd939

What platforms do you use to access UI ?

Windows

What browsers do you use to access the UI ?

Mozilla Firefox

Command Line Arguments

--vae-path models/Stable-diffusion/newVAE.vae.pt --autolaunch

Additional information, context and logs

Freshly created embedding: testembed.zip

Log:

Error verifying pickled file from C:\Users\demos\Documents\workspace\stable-diffusion-webui\embeddings\testembed.pt:
Traceback (most recent call last):
  File "C:\Users\demos\Documents\workspace\stable-diffusion-webui\modules\safe.py", line 97, in load
    check_pt(filename)
  File "C:\Users\demos\Documents\workspace\stable-diffusion-webui\modules\safe.py", line 77, in check_pt
    check_zip_filenames(filename, z.namelist())
  File "C:\Users\demos\Documents\workspace\stable-diffusion-webui\modules\safe.py", line 69, in check_zip_filenames
    raise Exception(f"bad file inside {filename}: {name}")
Exception: bad file inside C:\Users\demos\Documents\workspace\stable-diffusion-webui\embeddings\testembed.pt: testembed/data.pkl


The file may be malicious, so the program is not going to read it.
You can skip this check with --disable-safe-unpickle commandline argument.


Error loading emedding testembed.pt:
Traceback (most recent call last):
  File "C:\Users\demos\Documents\workspace\stable-diffusion-webui\modules\textual_inversion\textual_inversion.py", line 133, in load_textual_inversion_embeddings
    process_file(fullfn, fn)
  File "C:\Users\demos\Documents\workspace\stable-diffusion-webui\modules\textual_inversion\textual_inversion.py", line 103, in process_file
    if 'string_to_param' in data:
TypeError: argument of type 'NoneType' is not iterable

[msoucy]

The only thing I can think of that might be unusual about my setup is that I changed the pip install command in launch.py to the following:

pip install torch==1.13.0+cu116 torchvision==0.14.0+cu116 --extra-index-url https://download.pytorch.org/whl/cu116

(I was trying to squeeze more performance out of my 4090)

[takanorix]

Using torch 1.13.0+cu117, torchvision 0.14.0+cu117 for speedup and encountered the same problem. I think you can avoid your problem for now by reverting back to the original torch 1.12.1+cu113 and torchvision 0.13.1+cu113.

[nachoal]

Is there any configuration we can add to the UI loader to avoid this issue?

[kamalWam]

I noticed the same issue after selecting "Create hypernetwork" and using the default values. Once I select "Train" -> "Train Hypernetwork", it fails loading the .pt it just created with error below. Can bypass that with --disable-safe-unpickle

This is running on Mac M1

file path: /stable-diffusion-webui-mps-master/stable-diffusion-webui/models/hypernetworks/hyper1.pt Error verifying pickled file from /stable-diffusion-webui-mps-master/stable-diffusion-webui/models/hypernetworks/hyper1.pt: Traceback (most recent call last): File "/stable-diffusion-webui-mps-master/stable-diffusion-webui/modules/safe.py", line 97, in load check_pt(filename) File "/stable-diffusion-webui-mps-master/stable-diffusion-webui/modules/safe.py", line 77, in check_pt check_zip_filenames(filename, z.namelist()) File "/stable-diffusion-webui-mps-master/stable-diffusion-webui/modules/safe.py", line 69, in check_zip_filenames raise Exception(f"bad file inside {filename}: {name}") Exception: bad file inside /stable-diffusion-webui-mps-master/stable-diffusion-webui/models/hypernetworks/hyper1.pt: hyper1/data.pkl

The file may be malicious, so the program is not going to read it. You can skip this check with --disable-safe-unpickle commandline argument.

[aliencaocao]

Same issue with torch 11.3+cu117 on Windows.

[richardstrnad]

Using torch 1.13.0+cu117, torchvision 0.14.0+cu117 for speedup and encountered the same problem. I think you can avoid your problem for now by reverting back to the original torch 1.12.1+cu113 and torchvision 0.13.1+cu113.

I wasted so much time today, because I somehow upgraded these packages without realizing an broke my dreambooth training... thanks!!

[nachoal]

I ended up modifying the startup script with --disable-safe-unpickle for now as: python webui.py --disable-safe-unpickle --precision full --no-half --use-cpu Interrogate GFPGAN CodeFormer $@

Of course I'm using a trusted model from my own build

[ye7iaserag]

Can confirm happening to me too with official stable diffusion 1.5 model files

[OedoSoldier]

Same, torch 1.13+cu117

Need to add --disable-safe-unpickle

[smirkingface]

This is an actual issue with the safety checker. It assumes the internal structure of any checkpoint is 'archive/data.pkl', but any model saved directly with torch.save(model, 'filename.ckpt') will have the structure 'filename/data.pkl'. The only reason many checkpoints get 'archive' instead, is that pytorch-lightning does not save directly to a file, but uses an intermediate buffer.

Edit: Looked into it a bit further, this seems to be an issue with newer versions of pytorch. Checkpoints I created fairly recently still used 'archive/data.pkl'.

[smirkingface]

This issue will be fixed with the following pull request: https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/5327