In which config (or in None) should the `predict_epsilon` config be saved?

[natolambert]

We're discussing where certain properties should be saved in this repository. For example, this predict_epsilon property is used to distinguish if a model is trained to directly reconstruct the original sample or the noise associated with it.

(used in this colab)

There are multiple arguments for where it could fall, and it is an interesting discussion when creating a library that is modular and useful at different levels of abstraction. Here are the different places the predict_epsilon property could be stored

• model config: here the user could load a pre-trained model and instantly know to use it. Though, there is an argument to be made that the best model classes will be data-agnostic and this is related to the training structure.
• pipeline config: here the config will automatically handle the sampling, likely without the user knowing, and it will happen naturally. This is likely best if models are going to be used primarily for inference
• scheduler config: this is the last natural option and operates at a sort of conceptual middle ground. The scheduler is where is sampling actually occurs, so maybe that is the best spot for the config.

Important context is that the library is currently heavily used for inference, but hopefully will be used for training as well. Some of the positioning of things like predict_epsilon will be more logical in a training script rather than inference.

[patrickvonplaten]

@patil-suraj @anton-l what do you think?

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[patrickvonplaten]

Let's close this for now and discuss on PR once we go multi-modality

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.