statistics icon indicating copy to clipboard operation
statistics copied to clipboard

Problems with Student-T distribution

Open Shimuuar opened this issue 6 years ago • 1 comments

So far following problems with Student-T distribution were identified:

  1. quantile lose precision for p≈0.5 since x becomes close to 1, while p is near zero
quantile :: StudentT -> Double -> Double
quantile (StudentT ndf) p
  | p >= 0 && p <= 1 =
    let x = invIncompleteBeta (0.5 * ndf) 0.5 (2 * min p (1 - p))
    in case sqrt $ ndf * (1 - x) / x of
         r | p < 0.5   -> -r
           | otherwise -> r
  1. Both precision and performance suffer greatly for large degrees of freedom. Reason is likely incomplete beta and its inverse perform poorly for large parameters. However distribution becomes close to normal so other approximations could work!

Shimuuar avatar Jan 01 '20 19:01 Shimuuar

RE 2.

Quick experimentation with mpmath shows that precision becomes worse for larger NDF. (N of ulps of error proportional to NDF). For very large N mpmath fails itself which I think doesn't bring any good news for algorithm being used...

Shimuuar avatar Jan 01 '20 20:01 Shimuuar