Confusion about the use of from_config and its relation to _use_default_values

[christopher-beckham]

Describe the bug

I'm not sure if this qualifies as a bug because it depends on what the intent is behind the code. At the very least, I would classify it as peculiar behaviour. Suppose I instantiate some scheduler (doesn't matter which one) and extract (a copy of) its config to modify after:

sched = EulerDiscreteScheduler()
config = dict(sched.config)

If I modify config to have different key/vals, e.g config['beta_end'] = 0.5, and then instantiate a new scheduler from that config:

sched2 = EulerDiscreteScheduler.from_config(config)

sched2 is identical to sched, and this is also reflected by looking at the string repr of the object (i.e. beta_end is still the default value). I didn't understand why any changes in config weren't being reflected in sched2 until I realised that config contains an extra list called _use_default_values, which appears to track what kwargs were not passed into the constructor. In our case it's everything since we just used sched = EulerDiscreteScheduler():

['beta_end', 'sigma_max', 'timestep_spacing', 'interpolation_type', 'steps_offset', 'timestep_type', 'trained_betas', 'beta_start', 'sigma_min', 'beta_schedule', 'use_karras_sigmas', 'num_train_timesteps', 'prediction_type']

which means that beta_end gets ignored as per the behaviour in extract_init_dict (class ConfigMixin).

Ok so that means if we do the following:

# pass in explicit kwarg, beta_end=0.4
sched = EulerDiscreteScheduler(beta_end=0.4)
config = dict(sched.config)
config['beta_end'] = 0.5
sched2 = EulerDiscreteScheduler.from_config(config)
# this will be 0.5
print(sched2.beta_end)

then things work just fine.

I don't quite understand the intent behind this. Sure, I could work around the issue by actually instantiating the scheduler like sched2 = EulerDiscreteScheduler(**config) and that works, but I am still confused about the above because the docs describe from_config as "Instantiate a Python class from a config dictionary.", almost as if using <classname>.from_config is synonymous with instantiating a class via its constructor with <classname>(**kwargs). However, the way it works now is that config acts like a quasi-immutable dictionary (from the pov of the new scheduler object, assuming _use_default_values is inside).

Can we discuss what the intent is here? Thanks.

Reproduction

See above.

Logs

See above.

System Info

Latest version as of now.

Who can help?

No response

[DN6]

cc: @yiyixuxu for visibility

[christopher-beckham]

@yiyixuxu also cc'ing, thanks!

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[huyduong7101]

I faced the similar issue when I customized a UNetMotionModel. When I tried to pass the parameter "mid_block_type" into UNetMotionModel.from_config, but it used the default value of "mid_block_type". That's so weird and tricky.

[christopher-beckham]

retagging @yiyixuxu again for visibility, would be curious if there was a particular design decision behind this even if you don't have time to go deeply into the code, thanks!

[yiyixuxu]

hi @christopher-beckham hope this PR can provide some context on why it is designed this way :) https://github.com/huggingface/diffusers/pull/3929#issuecomment-1618919655

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.