software-review icon indicating copy to clipboard operation
software-review copied to clipboard

Published 20 hours ago • verified publisher icon ropensci

priorsense - Prior Diagnostics and Sensitivity Analysis

Open n-kall opened this issue 6 months ago • 7 comments
trafficstars

Submitting Author Name: Noa Kallioinen Submitting Author Github Handle: @n-kall Other Package Authors Github handles: (comma separated, delete if none) @topipa, @paul-buerkner, @avehtari Repository: https://github.com/n-kall/priorsense Version submitted: 1.1.1.9000 Submission type: Stats Badge grade: bronze/silver/gold (select one) Editor: @emitanaka Reviewers: TBD

Archive: TBD Version accepted: TBD Language: en

  • Paste the full DESCRIPTION file inside a code block below:
Package: priorsense
Title: Prior Diagnostics and Sensitivity Analysis
Version: 1.1.1.9000
Authors@R: c(person("Noa", "Kallioinen", email = "[email protected]", role = c("aut", "cre", "cph")),
	     person("Topi", "Paananen", role = c("aut")),
	     person("Paul-Christian", "Bürkner", role = c("aut")),
	     person("Aki", "Vehtari", role = c("aut")),
	     person("Frank", "Weber", role = c("ctb"))
	     )
Description: Provides functions for prior and likelihood sensitivity analysis in Bayesian models. Currently it implements methods to determine the sensitivity of the posterior to power-scaling perturbations of the prior and likelihood.
License: GPL (>= 3)
Encoding: UTF-8
LazyData: true
Roxygen: list(markdown = TRUE, roclets = c ("namespace", "rd", "srr::srr_stats_roclet"))
RoxygenNote: 7.3.2
Imports:
    checkmate (>= 2.3.1),
    ggdist (>= 3.3.2),
    ggh4x (>= 0.2.5),
    ggplot2 (>= 3.5.1),
    grDevices (>= 3.6.2),
    matrixStats (>= 1.3.0),
    posterior (>= 1.6.0),
    rlang (>= 1.1.4),
    stats,
    tibble (>= 3.2.1),
    utils
Suggests:
    bayesplot (>= 1.11.1),
    brms (>= 2.22.0),
    cmdstanr (>= 0.8.1),
    iwmm (>= 0.0.1),
    philentropy (>= 0.8.0),
    quarto (>= 1.4.4),
    R2jags (>= 0.8),
    rstan (>= 2.32.6),
    testthat (>= 3.0.0),
    transport (>= 0.15),
    vdiffr (>= 1.0.8)
Config/testthat/edition: 3
Depends:
    R (>= 3.6.0)
VignetteBuilder: quarto
Additional_repositories:
    https://topipa.r-universe.dev,
    https://stan-dev.r-universe.dev
URL: https://github.com/n-kall/priorsense, https://n-kall.github.io/priorsense/
BugReports: https://github.com/n-kall/priorsense/issues
Config/Needs/website: quarto

Scope

  • Please indicate which of our statistical package categories this package falls under. (Please check one or more appropriate boxes below):

    Statistical Packages

    • [ ] Bayesian and Monte Carlo Routines
    • [ ] Dimensionality Reduction, Clustering, and Unsupervised Learning
    • [ ] Machine Learning
    • [ ] Regression and Supervised Learning
    • [x] Exploratory Data Analysis (EDA) and Summary Statistics
    • [ ] Spatial Analyses
    • [ ] Time Series Analyses
    • [ ] Probability Distributions

Pre-submission Inquiry

  • [x] A pre-submission inquiry has been approved in issue#697

General Information

  • Who is the target audience and what are scientific applications of this package?

The target audience is users of Bayesian models who would like to check the prior (and likelihood) sensitivity of their model. This can demonstrate the robustness of the results, as well as how important the prior choice is.

  • Paste your responses to our General Standard G1.1 here, describing whether your software is:

    • The first implementation of a novel algorithm; or
    • The first implementation within R of an algorithm which has previously been implemented in other languages or contexts; or
    • An improvement on other implementations of similar algorithms in R.

Response:

The \pkg{priorsense} package provides functions for prior and likelihood sensitivity analysis of Bayesian models. Currently it implements methods to determine the sensitivity of the posterior to power-scaling perturbations of the prior and likelihood and is the first implementation of the method described in Kallioinen et al. (2023).

Response: Not applicable

Badging

Gold

  • If aiming for silver or gold, describe which of the four aspects listed in the Guide for Authors chapter the package fulfils (at least one aspect for silver; three for gold)

  • Compliance with a good number of standards beyond those identified as minimally necessary

  • Have a demonstrated generality of usage beyond one single envisioned use case: While originally developed for Stan models, priorsense has been extended to work with brms models, JAGS models and arbitrary posterior draws.

  • Demonstrating excellence in compliance with multiple standards from at least two broad sub-categories: Aiming for excellence in documentation (6.1.1 and 6.3.1) with examples and vignettes, and (6.3.5) Visualization and summarization output

Technical checks

Confirm each of the following by checking the box.

This package:

Publication options

  • [x] Do you intend for this package to go on CRAN?
  • [ ] Do you intend for this package to go on Bioconductor?

Response: priorsense is already on CRAN

Code of conduct

  • [x] I agree to abide by rOpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.

n-kall avatar May 05 '25 18:05 n-kall

Thanks for submitting to rOpenSci, our editors and @ropensci-review-bot will reply soon. Type @ropensci-review-bot help for help.

ropensci-review-bot avatar May 05 '25 18:05 ropensci-review-bot

:rocket:

The following problem was found in your submission template:

  • 'statsgrade' variable must be one of [bronze, silver, gold] Editors: Please ensure these problems with the submission template are rectified. Package checks have been started regardless.

:wave:

ropensci-review-bot avatar May 05 '25 18:05 ropensci-review-bot

Checks for priorsense (v1.1.1.9000)

git hash: cbeed8fd

  • :heavy_check_mark: Package is already on CRAN.
  • :heavy_check_mark: has a 'codemeta.json' file.
  • :heavy_check_mark: has a 'contributing' file.
  • :heavy_check_mark: uses 'roxygen2'.
  • :heavy_check_mark: 'DESCRIPTION' has a URL field.
  • :heavy_check_mark: 'DESCRIPTION' has a BugReports field.
  • :heavy_check_mark: Package has at least one HTML vignette
  • :heavy_check_mark: All functions have examples.
  • :heavy_check_mark: Package has continuous integration checks.
  • :heavy_check_mark: Package coverage is 81.8%.
  • :heavy_check_mark: R CMD check found no errors.
  • :heavy_check_mark: R CMD check found no warnings.

Package License: GPL (>= 3)

1. rOpenSci Statistical Standards (srr package)

This package is in the following category:

  • Exploratory Data Analysis

:heavy_check_mark: All applicable standards [v0.2.0] have been documented in this package (104 complied with; 22 N/A standards)

Click to see the report of author-reported standards compliance of the package with links to associated lines of code, which can be re-generated locally by running the srr_report() function from within a local clone of the repository.

2. Package Dependencies

Details of Package Dependency Usage (click to open)

The table below tallies all function calls to all packages ('ncalls'), both internal (r-base + recommended, along with the package itself), and external (imported and suggested packages). 'NA' values indicate packages to which no identified calls to R functions could be found. Note that these results are generated by an automated code-tagging system which may not be entirely accurate.

type package ncalls
internal base 392
internal priorsense 121
internal graphics 3
imports checkmate 86
imports stats 80
imports posterior 30
imports ggplot2 27
imports utils 16
imports grDevices 6
imports matrixStats 3
imports tibble 2
imports ggh4x 1
imports ggdist NA
imports rlang NA
suggests transport 5
suggests philentropy 3
suggests brms 2
suggests iwmm 2
suggests bayesplot NA
suggests cmdstanr NA
suggests quarto NA
suggests R2jags NA
suggests rstan NA
suggests testthat NA
suggests vdiffr NA
linking_to NA NA

Click below for tallies of functions used in each package. Locations of each call within this package may be generated locally by running 's <- pkgstats::pkgstats(<path/to/repo>)', and examining the 'external_calls' table.

base

c (84), length (41), list (40), return (35), for (13), sum (12), log (10), data.frame (8), min (8), paste0 (8), unique (8), is.null (6), mean (6), sub (6), max (5), cumsum (4), order (4), seq_along (4), sqrt (4), vector (4), ceiling (3), diag (3), exp (3), labels (3), mode (3), q (3), round (3), scale (3), seq_len (3), switch (3), abs (2), args (2), as.data.frame (2), attr (2), factor (2), get (2), lapply (2), levels (2), rep (2), seq (2), setdiff (2), subset (2), which (2), as.array (1), as.character (1), as.factor (1), as.numeric (1), colnames (1), diff (1), dim (1), drop (1), eigen (1), if (1), ifelse (1), is.numeric (1), merge (1), numeric (1), plot (1), pmax (1), pmin (1), rbind (1), rev (1), rle (1), rowSums (1), sort (1), summary (1), t (1), tcrossprod (1), unname (1)

priorsense

create_priorsense_data (6), get_powerscaling_details (6), pareto_k_mean (6), pareto_k_var (6), measure_divergence (5), cjs_dist (3), ess_mean (3), ess_var (3), hellinger_dist (3), js_dist (3), js_div (3), kl_dist (3), kl_div (3), ks_dist (3), powerscale_sequence (3), remove_unwanted_vars (3), whiten_draws (3), cdf_fun (2), create_priorsense_data.CmdStanFit (2), create_priorsense_data.stanfit (2), powerscale_examples (2), powerscale_gradients (2), prepare_plot (2), prepare_plot_data (2), quantile_weighted (2), above_one_comparison (1), below_one_comparison (1), compute_group (1), create_priorsense_data.default (1), create_priorsense_data.draws (1), create_priorsense_data.rjags (1), default_priorsense_colors (1), ewcdf (1), example_powerscale_model (1), find_alpha_threshold (1), find_alpha_threshold.default (1), find_alpha_threshold.priorsense_data (1), get_draws_CmdStanFit (1), get_draws_stanfit (1), is_constant (1), log_lik_draws (1), log_lik_draws.CmdStanFit (1), log_lik_draws.draws (1), log_lik_draws.stanfit (1), log_prior_draws (1), log_prior_draws.CmdStanFit (1), log_prior_draws.draws (1), log_prior_draws.stanfit (1), mad_weighted (1), mean_weighted (1), median_weighted (1), mv_kl_div (1), mv_wasserstein_dist (1), plot.priorsense_plot (1), powerscale (1), powerscale_derivative (1), powerscale_divergence_gradients (1), powerscale_gradients.default (1), powerscale_gradients.priorsense_data (1), powerscale_log_ratio_fun (1), powerscale_plot_dens (1), powerscale.default (1), powerscale.priorsense_data (1), rowsums_draws (1), scaled_log_ratio (1)

checkmate

assert_numeric (47), assertCharacter (7), assert_number (6), assert_vector (6), assert_logical (5), assertNumber (4), assert_character (2), assert_choice (2), assertChoice (2), assertNumeric (2), assertClass (1), assertFunction (1), assertLogical (1)

stats

weights (37), dist (8), density (6), loadings (6), sd (6), var (6), mad (2), median (2), approxfun (1), cov2cor (1), ecdf (1), ks.test (1), reshape (1), sigma (1), terms (1)

posterior

summarise_draws (9), subset_draws (6), pareto_khat (2), weight_draws (2), as_draws_array (1), as_draws_df (1), as_draws_matrix (1), bind_draws (1), merge_chains (1), mutate_variables (1), nchains (1), ndraws (1), quantile2 (1), resample_draws (1), variables (1)

ggplot2

aes (7), vars (6), element_blank (3), after_stat (2), ggplot (2), labeller (2), theme (2), ggproto (1), guide_legend (1), stat_ecdf (1)

utils

data (13), head (1), tail (1), vi (1)

grDevices

colors (5), colours (1)

transport

subwasserstein (2), wpp (2), wasserstein1d (1)

graphics

title (2), points (1)

matrixStats

weightedMad (1), weightedMean (1), weightedMedian (1)

philentropy

hellinger (1), jensen_shannon (1), kullback_leibler_distance (1)

brms

brmsterms (2)

iwmm

constrain_draws (1), moment_match (1)

tibble

as_tibble_col (2)

ggh4x

facetted_pos_scales (1)

NOTE: Some imported packages appear to have no associated function calls; please ensure with author that these 'Imports' are listed appropriately.

3. Statistical Properties

This package features some noteworthy statistical properties which may need to be clarified by a handling editor prior to progressing.

Details of statistical properties (click to open)

The package has:

  • code in R (100% in 29 files) and
  • 4 authors
  • no vignette
  • no internal data file
  • 11 imported packages
  • 37 exported functions (median 9 lines of code)
  • 174 non-exported functions in R (median 13 lines of code)

Statistical properties of package structure as distributional percentiles in relation to all current CRAN packages The following terminology is used:

  • loc = "Lines of Code"
  • fn = "function"
  • exp/not_exp = exported / not exported

All parameters are explained as tooltips in the locally-rendered HTML version of this report generated by the checks_to_markdown() function

The final measure (fn_call_network_size) is the total number of calls between functions (in R), or more abstract relationships between code objects in other languages. Values are flagged as "noteworthy" when they lie in the upper or lower 5th percentile.

measure value percentile noteworthy
files_R 29 88.3
files_inst 1 94.9
files_vignettes 0 0.0 TRUE
files_tests 15 92.2
loc_R 3050 89.1
loc_inst 28 23.8
loc_tests 770 78.9
num_vignettes 0 0.0 TRUE
n_fns_r 211 89.4
n_fns_r_exported 37 82.0
n_fns_r_not_exported 174 90.7
n_fns_per_file_r 4 63.6
num_params_per_fn 4 51.1
loc_per_fn_r 12 36.5
loc_per_fn_r_exp 9 19.7
loc_per_fn_r_not_exp 13 43.0
rel_whitespace_R 22 91.4
rel_whitespace_inst 29 29.1
rel_whitespace_tests 20 77.5
doclines_per_fn_exp 93 88.8
doclines_per_fn_not_exp 0 0.0 TRUE
fn_call_network_size 106 79.3

3a. Network visualisation

Click to see the interactive network visualisation of calls between objects in package

4. goodpractice and other checks

Details of goodpractice checks (click to open)

3a. Continuous Integration Badges

R-CMD-check

GitHub Workflow Results

id name conclusion sha run_number date
14618185403 pages build and deployment success 979a6f 28 2025-04-23
14467436006 pkgcheck failure dbe734 1 2025-04-15
14617964150 pkgdown success cbeed8 103 2025-04-23
14617964116 R-CMD-check success cbeed8 513 2025-04-23

3b. goodpractice results

R CMD check with rcmdcheck

rcmdcheck found no errors, warnings, or notes

Test coverage with covr

Package coverage: 81.83

Cyclocomplexity with cyclocomp

The following functions have cyclocomplexity >= 15:

function cyclocomplexity
powerscale_sequence.priorsense_data 28
powerscale.priorsense_data 22
powerscale_plot_ecdf.powerscaled_sequence 19
powerscale_plot_dens.powerscaled_sequence 18
powerscale_gradients.priorsense_data 17
prepare_plot_data 16

Static code analyses with lintr

lintr found the following 82 potential issues:

message number of times
Avoid library() and require() calls in packages 13
Lines should not be more than 80 characters. This line is 103 characters. 1
Lines should not be more than 80 characters. This line is 109 characters. 3
Lines should not be more than 80 characters. This line is 110 characters. 9
Lines should not be more than 80 characters. This line is 118 characters. 2
Lines should not be more than 80 characters. This line is 130 characters. 1
Lines should not be more than 80 characters. This line is 132 characters. 1
Lines should not be more than 80 characters. This line is 146 characters. 1
Lines should not be more than 80 characters. This line is 147 characters. 1
Lines should not be more than 80 characters. This line is 152 characters. 1
Lines should not be more than 80 characters. This line is 165 characters. 1
Lines should not be more than 80 characters. This line is 81 characters. 1
Lines should not be more than 80 characters. This line is 82 characters. 4
Lines should not be more than 80 characters. This line is 83 characters. 2
Lines should not be more than 80 characters. This line is 85 characters. 3
Lines should not be more than 80 characters. This line is 86 characters. 3
Lines should not be more than 80 characters. This line is 88 characters. 1
Lines should not be more than 80 characters. This line is 90 characters. 11
Lines should not be more than 80 characters. This line is 91 characters. 2
Lines should not be more than 80 characters. This line is 94 characters. 1
Lines should not be more than 80 characters. This line is 95 characters. 6
Lines should not be more than 80 characters. This line is 96 characters. 3
Lines should not be more than 80 characters. This line is 99 characters. 11
Package Versions

package version
pkgstats 0.2.0.54
pkgcheck 0.1.2.126
srr 0.1.4.4

Editor-in-Chief Instructions:

This package is in top shape and may be passed on to a handling editor

ropensci-review-bot avatar May 05 '25 18:05 ropensci-review-bot

@n-kall Thanks for this exciting submission. We're hoping to find a stats editor to handle it in the coming days. You'll see progress as soon as that has happened.

mpadge avatar May 20 '25 08:05 mpadge

@ropensci-review-bot assign @emitanaka as editor

mpadge avatar May 22 '25 07:05 mpadge

Assigned! @emitanaka is now the editor

ropensci-review-bot avatar May 22 '25 07:05 ropensci-review-bot

@n-kall Please note that @emitanaka will act as editor for your submission, but she will be away until early June. You'll have to wait until then before things proceed further. Sorry for any inconvenience, and thanks in advance for bearing with us.

mpadge avatar May 22 '25 07:05 mpadge

Hi @mpadge and @emitanaka, I was just wondering what the status of this submission is? I see it is tagged as "holding", is there anything needed from my side at this stage?

n-kall avatar Sep 04 '25 12:09 n-kall

I'm currently doing the EiC role and will try to get things rolling again. I wasn't sure on the holding status either. Sorry about that!

ldecicco-USGS avatar Sep 04 '25 12:09 ldecicco-USGS

Hi @n-kall, please accept apologies once again for long delays here. All of our editors are volunteers, and sometimes the world gets in the way. We're doing our best to get things moving 🚴

mpadge avatar Nov 10 '25 16:11 mpadge