How to pick sigma?

[ex3ndr]

Hey everyone, i am tryin to figure out what values of σ aka sigma is meant to be used during training? There are no mentioning of a specific value in papers for some reason.

[zvorinji]

The paper does mention 0.00001 as a starting point but unclear if that’s actually what they used. That said if testing out whether a different number could make the model converge faster, I’d go up (not down) and by orders of magnitude each test, and wouldn’t go above 1. So basically test 0.0001, if better than default, go test 0.001, and so on.

[ex3ndr]

Where did you get this number? I don't see it in the paper

[Subuday]

@ex3ndr Flow Matching paper recommends to set it sufficiently small. Guys from MatchaTTS(which also uses Flow Matching) trained their model with 1e-5 value.