Bad sample sigma scheduler

[lockmatrix]

Describe the bug Euler, step 25

I try to print the sigma (the difference of sigmas is something likes learning rate) in https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/be1596ce30b1ead6998da0c62003003dcce5eb2c/modules/sd_samplers.py#L389

get:

[14.6116, 11.4338,  9.0777,  7.3021,  5.9490,  4.9028,  4.0869,  3.4386,
         2.9182,  2.4957,  2.1488,  1.8592,  1.6160,  1.4080,  1.2278,  1.0712,
         0.9320,  0.8083,  0.6952,  0.5911,  0.4938,  0.3994,  0.3062,  0.2033,
         0.0313,  0.0000]

the difference of sigma:

dt: [-3.177830696105957,
 -2.356069564819336, 
-1.7756867408752441, 
-1.3530864715576172, 
-1.0462112426757812, 
-0.8158211708068848, 
-0.6483118534088135, 
-0.5204684734344482, 
-0.42246341705322266, 
-0.34693169593811035, 
-0.2895848751068115, 
-0.2432178258895874, 
-0.20793068408966064, 
-0.18019843101501465, 
-0.15663456916809082, 
-0.13917803764343262, 
-0.12373453378677368, 
-0.11305737495422363, 
-0.10407549142837524, 
-0.09735357761383057, 
-0.09442031383514404, 
-0.09321478009223938, 
-0.10290129482746124,  ***********
-0.17199255526065826,  ***********
-0.0312652550637722]

Last 3 dt seems abnormal, I think this may be a scheduler bug.

I'm trying to find a simple way to make it produce a more smooth dt curve

[lockmatrix]

the sum of dt (likes learning rate) is always -14.6116, the last dt is always -0.03126, the curve of sigma controls the distribution of dt.

ideas to impove:

• the last dt is always -0.03126 is not a good idea, becasue when step_num is large, the abs(dt) for second half of steps is smaller than 0.03126.
• the old sigma curve dropping fast near step_num, it leads to large abs(dt) near step_num, it's not a good idea.
• use a exp(-a step_idx) like function(which is called g(s) in following, assuming s start from 1) to replace the curve of sigma, it is simple and widely used.
• let abs(g(1) - g(step_num+1)) be 14.6116, to keep the sum of dt to 14.6116. or simply pin abs(g(1))=14.6116 if g(step_num+1) is small enough.
• do not let abs(g(step_num/2) - g(step_num+1)) too small to keep the tailing steps useful.

[C43H66N12O12S2]

the old sigma curve dropping fast near step_num, it leads to large abs(dt) near step_num, it's not a good idea.

I agree, but this is just how the SD scheduler works. Try the Karras variants.

[lockmatrix]

I agree, but this is just how the SD scheduler works. Try the Karras variants.

Karras has similar problem in last dt:

dt: [-1.3240442276000977, -1.1708478927612305, -1.0327138900756836, -0.9084315299987793, -0.7968769073486328, -0.6969814300537109, -0.6077454090118408, -0.5282387733459473, -0.4575979709625244, -0.39501309394836426, -0.3397289514541626, -0.2910478115081787, -0.24832558631896973, -0.21096330881118774, -0.17840701341629028, -0.1501503586769104, -0.12572622299194336, -0.10470682382583618, -0.08670148253440857, -0.07135394215583801, -0.05834062397480011, -0.04736843705177307, 
-0.03817273676395416, 
-0.030515573918819427,
-0.10000000149011612 **********
]

[C43H66N12O12S2]

Weird. Could you open an issue upstream at crowsonkb/k-diffusion?

[lockmatrix]

Weird. Could you open an issue upstream at crowsonkb/k-diffusion?

ok

[huige233]

stay https://github.com/CompVis/stable-diffusion/blob/main/ldm/models/diffusion/ddpm.py It can be seen that simga depends on the linear of the model_ Start and linear_ End, consider modifying it to 0.0005 and 0.02, and then there will be a more correct simga value

[catboxanon]

I'm not sure if this is a bug or if it's even related to kdiffusion or not, but have a bump. If it is then it should be patchable by the webui.