jasp-stats
[Feature Request]: Add / complete help files
last update for 0.95.2
Meta-bug: Collects missing/lacking help. Closed issues listed below are still relevant until marked by tick box.
One attempt to generate docs via ChatBot as a first is documented here https://github.com/jasp-stats/jasp-issues/issues/946
(almost) completely missing help files
- [ ] Reliability: Bland-Altman
- [ ] Frequencies: Classic+Bayesian Contingency Tables & Log-Linear Regression; Bayesian informed multinomial; Bayesian informed multi-binomial
- [ ] Learn Bayes: All
- [ ] Learn Stats: except the cloned distribution modules
- [ ] Bayesian Network Analysis
- [ ] Quality Control: Probability of Detection
Lacking help files
- [ ] t-Test Wilcox: several formulas available regarding tie handling, asymptotic vs exact (coin in R); document what formula used #452
- [ ] Bayesian t-Tests & BANOVAs, section plots, paragraph "prior and posterior" add: "The effect size delta is an estimate of the effect size distribution in the population. It is not a point estimate." from #1812 and "Credible interval: When this option is selected, the plot will contain central credible intervals based on flat prior distributions; the results will coincide with the frequentist confidence interval. The plot uses flat prior distributions because it shows a descriptive result that should not change depending on the details of the statistical model. The default percentage is set to 95%." from #3573
- [ ] BANOVA: add "descriptive plots are based on Jeffreys's parameterisation invariant priors used for estimation." from #195
- [ ] Correlation #1364
- [ ] Regression: Add note that polynomial / non-linear regression are available in "Visual Modeling" module
- [ ] Bayesian Regression: Explain why beta-binomial is default, see #1224
- [ ] Mediation analysis without "output" section & input section to short in explanations #1152
- [ ] Machine Learning: Random Forest Classification: Unclear what variable importance means. Is it permutation importance from Breiman (2001) 'Random Forests'? (from #747)
- [ ] Network Classic: Update Doc to reflect that Spearman method is gone (from #3470)
We have a project to generate Help file from the QML info properties: each option, section or group should have an info property with some translatable text. This will generate automatically the help md file, but also the ROxygene with R Syntax.
@tomtomme @boutinb yes I think using some AI generation is the way to go here (and I understand from some colleagues that we won't be unique in this) - it will simply be unmanageable to have all documentation written from scratch. I would suggest to have it included in the standard module development workflow, such that the analysis creator simply has to fact-check the generation documentation.
Just a note: If there is at some point a plan on pursuing the generative-AI part, then I suggest to look at this benchmark first: https://simple-bench.com/
It is quite a hard multiple-choice benchmark where even the best AI currently only reaches half the points a normal human could do.