39th president of the United States, probably
39th president of the United States, probably
You could always decompose the default function to have multiple other functions that are called inside it, which the user can also import and use in their functions if needed.
> actually would be nice to support list of callbacks since now we provide official ones that user can mix and match Yeah, I think this is the right way...
It's probably easier if I write it out in some pseudocode. Writing it down, I think `SamplingOutput` is probably redundant, so maybe we could come up with a better name...
> a bit late to the party here, but adding one use-case: modifying or skipping steps. right now, loop is fixed and no matter what happens in callbacks, they cannot...
> ohh thanks! for this PR we will keep it simple and support official callbacks with minimum change to our pipelines what you proposed will introduce a pretty drastic change...
Yeah, this would require even more re-engineering. You would need sampling functions to be a part of the scheduler, and all of them would need to be passed to the...
Why wouldn't you just subclass the scheduler and then overwrite the get timestep method? That seems trivial?
Opened as #7808
I can try to upload a pypi package of the fixed optimiser later today. It is located here: https://github.com/AmericanPresidentJimmyCarter/test-torch-bfloat16-vit-training/blob/main/adam_bfloat16/__init__.py#L22
fp16 is usually degraded and when you are training with it you need to use tricks like gradient scaling to work at all. For training scripts I am not sure...