scheduler_config.json

[rmac85]

I notice when I train a custom model with train_deambooth.py there are two different schedulers, a regular scheduler and a noise scheduler, but there is only one scheduler_config.json in my directory when finished. In this case it is DDIM and DDPM for the noise scheduler. Both of these schedulers have different beta_start and beta_end and also the DDPM has a variance_type option.

I have tried changing: noise_scheduler = DDPMScheduler.from_config(args.pretrained_model_name_or_path, subfolder="scheduler") to:
noise_scheduler = DDPMScheduler.from_config(args.pretrained_model_name_or_path, subfolder="scheduler", beta_start=0.0001, beta_end=0.02, variance_type="learned")

This is not throwing any error but I'm not sure it's really working or if it's conflicting in any way. I don't even know if it's necessary. Just a question if any knows more about how this works.

[patrickvonplaten]

@williamberman could you take a look here?

[williamberman]

@rmac85 thanks for the issue! I might need some more specifics on what you're asking.

I think one of the schedulers you're talking about is the scheduler in the trained model used in the training script here https://github.com/huggingface/diffusers/blob/4c52982a0be7dd850fb9eac55b11509846e4bbe6/examples/dreambooth/train_dreambooth.py#L604

What is the other scheduler you're referring to?

[rmac85]

The other one is the pipeline scheduler (DDIM) in lines 485 and 736: https://github.com/ShivamShrirao/diffusers/blob/main/examples/dreambooth/train_dreambooth.py

Basically what I'm asking, because when fine-tuning is finished I only have one scheduler_config.json, is if DDIM and DDPM are using the same config. According to huggingface docs both of these have different recommended settings as mentioned above.

[patrickvonplaten]

Hey @rmac85,

You can use the same scheduling_config.json file interchangeably with both scheduler classes

[williamberman]

Yes, this is to be expected. One of the schedulers is used to handle adding noise while training, the other is used for inference. The config that actually determines the noise schedule used is always the same