Training on the v.2.0 512 weights creates the wrong .yaml file

[CypherQube]

Training on the v.2.0 512 weights creates the wrong .yaml

After training on the v2.0 512 weights then testing the new weights it generates tan images exactly like it is using the v2-inference-v.yaml that is for the v2.0 768 weights not v2.0 512 weights. I deleted the created .yaml file duplicated v2-inference.yaml (which is for the 512 model) & renamed it for the newly created weights & bam the new dreambooth weights worked fine.

[d8ahazard]

I'll need to devise a way to detect whether or not the model is 512 or 768 and copy the proper config file. Ideally, the webui would just do this for us.

[CypherQube]

I'll need to devise a way to detect whether or not the model is 512 or 768 and copy the proper config file. Ideally, the webui would just do this for us.

Why not just add a checkbox for now?

[d8ahazard]

Because we shouldn't need it. I'm actively working on comparing the 512 and 768 checkpoints to see which keys we can use to figure this out. I'm already detecting whether it's a V2 model or not, finding one more key that indicates whether 512 or 768 shouldn't be too hard. The only difference in the config is v-prediction.

On Tue, Dec 6, 2022 at 2:30 PM CypherQube @.***> wrote:

I'll need to devise a way to detect whether or not the model is 512 or 768 and copy the proper config file. Ideally, the webui would just do this for us.

Why not just add a checkbox for now?

— Reply to this email directly, view it on GitHub https://github.com/d8ahazard/sd_dreambooth_extension/issues/432#issuecomment-1339970082, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMO4NCC4CMVFKT4KMWI7SLWL6O6VANCNFSM6AAAAAASVRFXFE . You are receiving this because you commented.Message ID: @.***>

[d8ahazard]

I'm having some difficulty reproducing the issue. I took the -v config file and loaded the 512 model with it, and had no weird outputs. Could you possibly provide a copy of the "bad" config file, versus your "good" one?

[FGhrawi]

I have the same issue, trained SD2.0 512 model only for it to load in V mode, checked the .yaml and indeed it seems the 768 one was used. Weird thing is, it generates images with the 768 .yaml and actually produced garbage when I put in the normal 512 eps mode .yaml. So I basically have a 512 reso model thats running in v-predict now, though the outputs are a bit strange (which made me look into it and realize what was happening). I just checked and made sure that I used the sd2-512 base model during training.

[FGhrawi]

512test.yaml.txt

[FGhrawi]

Removing parameterization: "v" from the .yaml file also makes the output become completely garbage (but not tan, just random), So it seems this 512 model only "functions" using v-mode now which shouldn't be the case. SD2-512 training is currently busted.

[leppie]

I have figured it out.

1. Create the model from 512-base-ema.ckpt
2. Change "prediction_type": "v_prediction", to "prediction_type": "epsilon", in scheduler_config.json
3. Start training
4. Once done, remove parameterization: "v" from the model.yaml file
5. Click Refresh models and load model
6. Generate images
7. PROFIT!!!

Some issues remain, after training, the prediction type is reverted to v_prediction again in scheduler_config.json.

[benedlore]

I have figured it out.

1. Create the model from `512-base-ema.ckpt`

2. Change `"prediction_type": "v_prediction",` to `"prediction_type": "epsilon",` in `scheduler_config.json`

3. Start training

4. Once done, remove `parameterization: "v"` from the model.yaml file

5. Click Refresh models and load model

6. Generate images

7. PROFIT!!!

Some issues remain, after training, the prediction type is reverted to v_prediction again in scheduler_config.json.

During training, if you have it generating preview images every X steps, the preview images will still be the broken images (like all brown/tan/whatever, still using v-prediction). Do you know if there is somewhere else where we can edit out v_prediction so that the preview images generated during training are useful/correct?

Edit: Actually, even after editing out the 'parameterization: "v"' line, it looks like my resulting trained model still is coming out wrong. A cool looking abstract image, but nothing to do with training images.

[github-actions[bot]]

This issue is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days

[CypherQube]

@d8ahazard Has this issue been solved?

[mediocreatmybest]

@d8ahazard Has this issue been solved?

Issue still exists.

[nykwil]

Does 512 just generate brown sample images for anyone else?

[CypherQube]

Does 512 just generate brown sample images for anyone else?

It's because the 512 weights are loading with v2-inference-v.yaml instead of the v2-inference.yaml, you can fix this by duplicating the v2-inference.yaml & renaming it to the name of the model you trained using the 512 model

[CypherQube]

@d8ahazard Why you can't just create a checkbox until you've solved how to detect the right .yaml file automatically?

[mediocreatmybest]

@d8ahazard Why you can't just create a checkbox until you've solved how to detect the right .yaml file automatically?

Another option would be to read the yaml file included with the model?

As currently we need to have a yaml file matched with the model with automatic1111.

[nykwil]

I don't understand where do I put the 512 yaml for training. There's no ckpt file in the working folder.

On Mon, Dec 26, 2022, 8:05 p.m. mediocreatmybest @.***> wrote:

@d8ahazard https://github.com/d8ahazard Why you can't just create a checkbox until you've solved how to detect the right .yaml file automatically?

Another option would be to read the yaml file included with the model?

As currently we need to have a yaml file matched with the model with automatic1111.

— Reply to this email directly, view it on GitHub https://github.com/d8ahazard/sd_dreambooth_extension/issues/432#issuecomment-1365595412, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHU46DBHREITZGM6C4AGE3WPJTHNANCNFSM6AAAAAASVRFXFE . You are receiving this because you commented.Message ID: @.***>

[mediocreatmybest]

I don't understand where do I put the 512 yaml for training. There's no ckpt file in the working folder. … On Mon, Dec 26, 2022, 8:05 p.m. mediocreatmybest @.> wrote: @d8ahazard https://github.com/d8ahazard Why you can't just create a checkbox until you've solved how to detect the right .yaml file automatically? Another option would be to read the yaml file included with the model? As currently we need to have a yaml file matched with the model with automatic1111. — Reply to this email directly, view it on GitHub <#432 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHU46DBHREITZGM6C4AGE3WPJTHNANCNFSM6AAAAAASVRFXFE . You are receiving this because you commented.Message ID: @.>

It is converted to a diffusers model, so it won't be a yaml file as such but json files within the working directory of the model you created.

stable-diffusion-webui --> models --> dreambooth --> YOURMODEL --> working These are the models you can select on the left after you have created them.

The main difference is: the scheduler for 768 version uses --> "prediction_type": "v_prediction" the scheduler for 512 version used --> "prediction_type": "epsilon"

To be honest I haven't had any luck getting 2.1 even close to training without crashes or CUDA errors or memory errors. So I have no idea if it is my venv install, or an issue with 2.x models, or an issue within the extension. So I'm not sure what else may be different between them.

I have no issues with 1.5 training just 2.x

[d8ahazard]

Yeeehaw!

This has been fixed in the latest version.

If a yaml file exists alongside the source checkpoint, it will be copied to the working dir and used when generating output checkpoints. If no config is found, the program will do it's best to infer the revision and size of the model.

[Straafe]

I am going to re-install the extension, but I was still getting tan preview sample images (every x steps/epochs) with SD2.1 512px, but class images generated perfect. With SD2.1 768px it failed to generate class images (it generated 1 pure black image and then stopped saving any class images but continued trying to train, so I cancelled it). I did have yaml files for both before trying, but maybe it is something wrong with my setup so I will fully reinstall.

[nykwil]

I can confirm this on the build last night

On Mon, Jan 2, 2023, 12:55 p.m. Straafe @.***> wrote:

I am going to re-install the extension, but I was still getting tan preview sample images (every x steps/epochs) with SD2.1 512px, but class images generated perfect. With SD2.1 768px it failed to generate class images (were pure black). I did have yaml files for both before trying, but maybe it is something wrong with my setup so I will fully reinstall.

— Reply to this email directly, view it on GitHub https://github.com/d8ahazard/sd_dreambooth_extension/issues/432#issuecomment-1369198469, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHU46HMLUASB4YWTWTEET3WQM6EPANCNFSM6AAAAAASVRFXFE . You are receiving this because you commented.Message ID: @.***>

[d8ahazard]

@leppie - You alive? Got a few questions regarding profiling for you, if you could be so kind to hop on Discord when you're free. ;)

[kakis2]

I think i have the same problem- getting brown images, and dreambooth not training on 2.1 512 version. Dreambooth revision: 9f4d931a319056c537d24669cb950d146d1537b0 SD-WebUI revision: 15e89ef0f6f22f823c19592a401b9e4ee477258c