the program is erasing/deleting checkpoints and loras by itself

[robbb45]

I'm completely terrified of using Stability Matrix again. A month ago, ALL my models were deleted. Erased. No trace. One of my Automatic1111 said something about the path didn't exist or something like that. and That was it, all gone. After a few updates, it apparently fixed this bug.

Now, after opening another SD installation, one of my LORAs folder was completely erased. I mean, if that was my creation folders, I would have lost a lot of originally trained/testing loras that was not backed up...

It is probably some kind of weird symbolic link bug. I have hard shell extension (to use symlink on windows easily) installed and I use it by myself a lot.

I'll try to explain in more details:

I have 3 A1111 installations.

1. 2 are 1.6 updated and they share the same venv folder by symbolic link created by me (to save on HD storage). Why? Because I want a clean SFW installation, so I can control what extensions and LORAs are in it. One is my main A1111, it uses the normal automatic created symlink created by Stability Matrix.
2. The second A1111 had the symlink created by Stability Matrix from the LORA and STABLEDISFFUSION checkpoints deleted by me and I created my own folder with my own symlink from the DATA from specific SFW checkpoints and Lora. This is the installations that a month ago DELETED all my checkpoints.
3. I have one 1.5 installation. for testing old broke extensions. This had the symlink created by Stability Matrix deleted and I created my own folder so only a specific folder is on the Lora folder of this installations. This specific lora folder is the one that got ERASED by stability matrix.

I'm checking it now and my actual Lora folder is in another HD (F:). with symlink to the DATA folder on StabilityMatrix (C:). And no, the deleted lora is not on the original folder, it got erased.

If I open the automatic1111 on the webui.bat it work normally. I just can't even think of using Stability Matrix again. It's a great tool, but deleting things like that with no notice or a way to recover it is completely absurd.

[ionite34]

Just clarifying some details to understand this, the shared folders under ./Data/Models are never deleted by us and we have no code path that delete any folders, symbolic or otherwise, in the shared directory.

If you elect for a package to be managed with shared folders, on each launch the targets will be created as Junctions (on Windows) linking to the shared folder sources. If it detects an existing symbolic folder or junction at the target with the same name, it will be deleted. Again this is only if you enable shared folder management for a package. If an existing real folder is at the target, all files will be moved to the shared folder before a new junction is created. If a moved file has the same name but different contents there should be an error message and we don't move or delete anything.

The second A1111 had the symlink created by Stability Matrix from the LORA and STABLEDISFFUSION checkpoints deleted by me and I created my own folder with my own symlink from the DATA from specific SFW checkpoints and Lora.

So these weren't the shared folders we manage, but file symbolic links you made manually?

I'm checking it now and my actual Lora folder is in another HD (F:). with symlink to the DATA folder on StabilityMatrix (C:)

It seems like you have a manual linking method set up, in this case I would recommend setting up the package without the shared folders option or disabling it in the menu.

[robbb45]

I'll try my best to clarify it. I don't really know much (I used to) about the difference from Symlink and junctions, here I'm calling the "folder symlink" as symlink. They might be junctions. I don't know.

The original files are stored on F:\Symlink AI\Lora
'\SFW is a real folder inside Lora. '\NSFW real folder '\My creation etc real folders

C:\stablediffusion1111\Data\Models\Lora\SFW this "SFW" folder is a symlink to a folder on > F:\Symlink AI\Lora\SFW The "Lora" folder is a actual folder. I have a bunch of other symlinks inside to > F. Just like the above example. Let's say "NSFW", "My Crations", etc.

C:\stablediffusion1111\Data\Packages\stable-diffusion-MAIN\models\Lora\SFW "Lora" folder here is a symlink, created by StabilityMatrix to > Data\Models\Lora. So naturally "SFW" is a symlink to > F as stated before

C:\stablediffusion1111\Data\Packages\SD15\models\Lora\SFW "Lora" here is NOT a symlink. It used to be, but I actually created a real folder. "SFW" inside this real folder (that has no other folder, just this) IS a symlink to F:\Symlink AI\Lora\SFW

I have a third package, kind of like the above. Isolating NSFW and other folders and keeping SFW and other folder I want. (those are not really the name of the folders, just to simplify).

OK. So, there is nothing anymore on F:\Symlink AI\Lora\SFW. It used to be. It was deleted somehow. I don't know how. I have a pretty good guess it happened after launching "SD15" package trough StabilityMatrix and it showing me an error and after that those Loras were not there anymore. The same thing happened a month ago, but instead of Lora, it was for "Data\models\Stable-diffusion" folder. I could test this, but I would need to backup all three packages because I'm afraid it will delete other things. I'll try to do it in the next few days.

It's nice that there is a option to manage all this by myself. I didn't know that. But still, it's kind of a crucial thing to not happen.

[ionite34]

C:\stablediffusion1111\Data\Packages\SD15\models\Lora\SFW "Lora" here is NOT a symlink. It used to be, but I actually created a real folder.

So creating real folders inside the managed folders like webui/models/Lora/... is going to cause them to be moved before deleting so that the managed shared link can be created, so this is not supported. If you want to have manual folders you'd need to not select shared folder option during the advanced install option or disable it with the 3-dot dropdown:

It seems we haven't anticipated for needing to move physical files that they themselves are symbolic sources for files or folders within the main shared folder, so it seems this may have caused deleting (since the contents of the files within the shared target and the source are the same). We will add an additional check to probably just error and stop early if any link sources appear in the shared target.

So currently if you place any Symlinks in the shared folder itself, that's perfectly safe, and placing them in the model folders with shared folder management enabled may cause deletion. If you'd like to manually place links in the package directory just ensure model sharing option is set to None.

[wwboynton]

I've got a fresh install on windows using webui-forge. I'm not using any symlinks. All my models were dropped into the appropriate directories within StabilityMatrix/Models/. I installed InvokeAI without launching it. I restarted the app after a few days of no use and ran the webui-forge, and all models were inexplicably gone. Thumbnails were still there.

After some inspection, I discovered they'd all been moved to StabilityMatrix\Packages\InvokeAI\invokeai-root\models\sd-1. Why this is, I do not understand. This is a really weird side effect kind of behaviour that shouldn't occur. Under no circumstances should StabilityMatrix or any of the packages relocate all of my models, especially not from the defined shared folder in the SM root into the model directories of an individual package.

Dropping this comment here since this issue is still open. If you think your models have been deleted, try poking around the files of any new packages you've installed.

[mohnjiles]

I've got a fresh install on windows using webui-forge. I'm not using any symlinks. All my models were dropped into the appropriate directories within StabilityMatrix/Models/. I installed InvokeAI without launching it. I restarted the app after a few days of no use and ran the webui-forge, and all models were inexplicably gone. Thumbnails were still there.

After some inspection, I discovered they'd all been moved to StabilityMatrix\Packages\InvokeAI\invokeai-root\models\sd-1. Why this is, I do not understand. This is a really weird side effect kind of behaviour that shouldn't occur. Under no circumstances should StabilityMatrix or any of the packages relocate all of my models, especially not from the defined shared folder in the SM root into the model directories of an individual package.

Dropping this comment here since this issue is still open. If you think your models have been deleted, try poking around the files of any new packages you've installed.

Hi, sorry about that, this one is actually a new "feature" of the latest InvokeAI. I would suggest also posting to their issue tracker as well if you'd like to see this behaviour change - https://github.com/invoke-ai/InvokeAI/issues

We are still working on a workaround on our end, but the easiest way to avoid this for now is to disable the Symlink option for Invoke & manually copy your models.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove the stale label or comment, else this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stale for 5 days with no activity.