anesthetic icon indicating copy to clipboard operation
anesthetic copied to clipboard

Modified Mann-Whitney-Wilcoxon U test for a uniform distribution of integers

Open JohannesBuchner opened this issue 11 months ago • 0 comments

A test for biased nested sampling was presented in section 4.5.2 of Buchner (2023, https://arxiv.org/abs/2101.09675).

This implements the same idea as https://arxiv.org/abs/2006.03371 except their KS test is sub-optimal because the variable (insertion order) is not continuous. Instead, this implements a modified Mann-Whitney-Wilcoxon U test, which also is in practice slightly more sensitive than the KS test.

A highly efficient implementation is achieved by keeping only a histogram of the insertion orders and comparing those to expectations from a uniform distribution. This also handles a variable number of live points.

Accumulator source code: https://johannesbuchner.github.io/UltraNest/_modules/ultranest/ordertest.html#infinite_U_zscore

See also https://johannesbuchner.github.io/UltraNest/index.html#licence

Approaches to application: (TBD) To quantify the convergence of a run, one route is to apply this test at the end of the run. Another approach is to reset the counters every time the test exceeds a z-score of 3 sigma, and report the run lengths, which quantify how many iterations nested sampling was able to proceed without detection of a insertion order problem.

JohannesBuchner avatar Jul 09 '23 10:07 JohannesBuchner