performance

performance copied to clipboard

Hi! Would it be possible to make your package compatible with {rethinking} models? Especially ulam, which is built on top of STAN. {tidybayes} package did the same, so it might be useful resource.

For example, this throws an error:

d <- list(x = dnorm(100))

m <- rethinking::ulam(
    alist(
        x ~ dnorm(mu, 1),
        mu ~ dnorm(0, 1)
    ), data = d
)

performance::check_model(m)

Thank you in advance.

The example isn't working, I assume alist() is not from rethinking?

alist() is base

Thank you, I fixed the code.

We can extract the raw rstan object from an "ulam" class object with:

d <- list(x = dnorm(100))

m <- rethinking::ulam(
    alist(
        x ~ dnorm(mu, 1),
        mu ~ dnorm(0, 1)
    ), data = d
)

m_stan <- m@stanfit

Then, we pass this to the various *.stanfit functions. Which it seems we don't currently support very well.

I'm not sure, I think ulma objects have no stanfit slot (I din't find one). For stanfit, we only provide basic support. I'm not sure if there are methods such as vcov(), df.residuals() or similar available for stanfit-objects?

ulam objects have a stanfit slot.

str(m, max.level = 2)
# Formal class 'ulam' [package "rethinking"] with 9 slots
#   ..@ call          : language rethinking::ulam(flist = alist(x ~ dnorm(mu, 1), mu ~ dnorm(0, 1)), data = d)
#   ..@ model         : chr "data{\n    real x;\n}\nparameters{\n    real mu;\n}\nmodel{\n    mu ~ normal( 0 , 1 );\n    x ~ normal( mu , 1 );\n}\n\n"
#   ..@ coef          : Named num -0.0265
#   .. ..- attr(*, "names")= chr "mean"
#   ..@ vcov          : num [1, 1] 0.578
#   ..@ data          :List of 1
#   ..@ start         :List of 1
#   ..@ pars          : chr "mu"
#   ..@ formula       :List of 2
#   ..@ formula_parsed:List of 7
#   ..$ stanfit   :Formal class 'stanfit' [package "rstan"] with 10 slots
#   ..$ generation: chr "ulam2018"
# 
m@stanfit
# Inference for Stan model: ulam_cmdstanr_c8f443e69bc7e945848df951bcfc2e6c-202204061135-1-3d534b.
# 1 chains, each with iter=1000; warmup=500; thin=1; 
# post-warmup draws per chain=500, total post-warmup draws=500.

#       mean se_mean   sd  2.5%   25%   50%   75% 97.5% n_eff Rhat
# mu   -0.03    0.06 0.76 -1.54 -0.56 -0.06  0.52  1.56   152    1
# lp__ -0.58    0.07 0.83 -2.95 -0.70 -0.30 -0.06  0.00   132    1

# Samples were drawn using NUTS(diag_e) at Wed Apr 06 11:35:27 AM 2022.
# For each parameter, n_eff is a crude measure of effective sample size,
# and Rhat is the potential scale reduction factor on split chains (at 
# convergence, Rhat=1).

rethinking provides a vcov slot for the ulam object and a few other methods. But, yes, ultimately full support for these models requires us to add first class support for Stan models, rather than the ad-hoc support we currently have by refitting rstanarm/brms models as frequentist models.