Associations.jl icon indicating copy to clipboard operation
Associations.jl copied to clipboard

Add `threshold` value to `SurrogateTest` and `LocalPermutationTest`

Open kahaaga opened this issue 1 year ago • 0 comments

A common use case for independence is to use SurrogateTest and LocalPermutationTest to test for independence between two or more variables using some information theoretic (conditional) independence measure.

Currently, for both SurrogateTest and LocalPermutationTest, in the summary printout, it is checked whether the estimated measure m_est computed on the original data exceeds some percentile of the estimated measure computed over an ensemble of shuffled input data. Here's one example of such a print-out.

It would be nice to specify some threshold value that, if m_est < threshold, the interpretation printed to the user is always that we can't reject the null hypothesis. For example, with transfer entropy, we know that the value should be non-negative, so any values that are non-negative should immediately lead to rejection of the null.

It is up to the analyst to decide whether to do this, so threshold should be a keyword to the tests themselves. It can for example be a Real or a Nothing (in which case no threshold is applied).

Skjermbilde 2023-10-02 kl  20 28 57

kahaaga avatar Oct 02 '23 18:10 kahaaga