drhead

The difference won't be very significant either way. It's technically incorrect, but I wouldn't worry about it too much. At least based on some of my ongoing experiments with adaptive...

> Thank you! I was getting underwhelming results from my experiments on zero terminal SNR models, and I figured there might be something wrong with the loss scaling. 😅 By...

@jihnenglin If you want to experiment with the learned timestep weighting, I have a prototype here as a gist: https://gist.github.com/drhead/ac6ecc1f6dc1fd478064f3d81ca12a25 It does typically take a long time with a fairly...

> Is the learning rate 0.005 good even for fine-tuning the entire model? I'm scared. Oh, no, no, you want that just for the loss weight MLP itself. It's using...

> @drhead This may be an irrelevant issue but most samplers give me weird results at lower (20-30) sampling steps for my v-prediction model. And they all work perfectly fine...

> But will it slow down the main generation? Having live preview enabled will always slightly slow down the main generation since it involves doing more work compared to not...

> It works on AMD! I've used cudastreams before with CuPy, so the support is there, and the CUDA checks will work on AMD. > > You will have to...

I do have it disabled. disable_nan_check would be causing a forced sync. Whatever is happening indicates that synchronizing the cuda stream doesn't do what it should on AMD.

> My installation stops delivering live previews as it was not enabled when having this patch on. No error was raised. Need more to go on than this. Are you...

> I am using cuda 3060 laptop with 3 steps on a 50 steps generation. Generation speed is approx 5 its. Tried Approx NN and TAESD but the result are...