rstan
rstan copied to clipboard
stan-dev
Issues with the experimental version of 2.26 from PR #887
https://github.com/stan-dev/rstan/pull/887 is huge and I think the PR thread will eventually become impossible to follow. So I am opening this issue where you can post anything that is missing in the alpha 2.26 version.
Thanks @hsbadr for working on this massive pull request!
I think the checklist for making this happen is as follows:
-
[ ] Check that it works for the early-adopters/devs on all OS-es for R4.x (R-release)
-
[ ] Check that it works for the early-adopters/devs on all OS-es for R3.x (R-oldrel)
I think after this we can merge the PR and deal with smaller issues later.
-
[ ] Make a beta-call on Discourse for other users with a disclaimer that it might break other packages that depend on rstan.
-
[ ] Hash out issues after the beta call
-
[ ] Try to get this to CRAN - this is where my knowledge stops. I expect this is where the "fun" will start.
so, the stanc() function seems? broken, in that it requires a file argument when, if you use the model_code argument, file should not be needed.
so, the stanc() function seems? broken, in that it requires a file argument when, if you use the model_code argument, file should not be needed.
stanc() interface and arguments didn't change. Can you provide a reproducible example?
library(rstan)
st <- '
functions{
real plus(real a){
return a + 1;
}
}
'
testcpp <- stanc(model_code = st)
Updated:
remove.packages(c("StanHeaders", "rstan")) install.packages("StanHeaders", repos = c("https://mc-stan.org/r-packages/", getOption("repos"))) install.packages("rstan", repos = c("https://mc-stan.org/r-packages/", getOption("repos")))
I installed using the above and then tried the following on my system (session info included below). This now seems to work :+1: Thank you @hsbadr ! Note that this is with the "broken" CRAN version of RcppParallel installed (see session info).
library(rstan)
#> Warning: package 'rstan' was built under R version 4.0.3
#> Loading required package: StanHeaders
#> Warning: package 'StanHeaders' was built under R version 4.0.3
#> rstan (Version 2.26.0.9000, GitRev: 2e1f913d3ca3)
#> For execution on a local, multicore CPU with excess RAM we recommend calling
#> options(mc.cores = parallel::detectCores()).
#> To avoid recompilation of unchanged Stan programs, we recommend calling
#> rstan_options(auto_write = TRUE)
#> Do not specify '-march=native' in 'LOCAL_CPPFLAGS' or a Makevars file
m <- stan_model(model_code = 'parameters {real y;} model {y ~ normal(0,1);}')
f <- sampling(m, iter = 100)
#>
#> SAMPLING FOR MODEL 'anon_model' NOW (CHAIN 1).
#> Chain 1:
#> Chain 1: Gradient evaluation took 1.1e-05 seconds
#> Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0.11 seconds.
#> Chain 1: Adjust your expectations accordingly!
#> Chain 1:
#> Chain 1:
#> Chain 1: WARNING: There aren't enough warmup iterations to fit the
#> Chain 1: three stages of adaptation as currently configured.
#> Chain 1: Reducing each adaptation stage to 15%/75%/10% of
#> Chain 1: the given number of warmup iterations:
#> Chain 1: init_buffer = 7
#> Chain 1: adapt_window = 38
#> Chain 1: term_buffer = 5
#> Chain 1:
#> Chain 1: Iteration: 1 / 100 [ 1%] (Warmup)
#> Chain 1: Iteration: 10 / 100 [ 10%] (Warmup)
#> Chain 1: Iteration: 20 / 100 [ 20%] (Warmup)
#> Chain 1: Iteration: 30 / 100 [ 30%] (Warmup)
#> Chain 1: Iteration: 40 / 100 [ 40%] (Warmup)
#> Chain 1: Iteration: 50 / 100 [ 50%] (Warmup)
#> Chain 1: Iteration: 51 / 100 [ 51%] (Sampling)
#> Chain 1: Iteration: 60 / 100 [ 60%] (Sampling)
#> Chain 1: Iteration: 70 / 100 [ 70%] (Sampling)
#> Chain 1: Iteration: 80 / 100 [ 80%] (Sampling)
#> Chain 1: Iteration: 90 / 100 [ 90%] (Sampling)
#> Chain 1: Iteration: 100 / 100 [100%] (Sampling)
#> Chain 1:
#> Chain 1: Elapsed Time: 0 seconds (Warm-up)
#> Chain 1: 0 seconds (Sampling)
#> Chain 1: 0 seconds (Total)
#> Chain 1:
#>
#> SAMPLING FOR MODEL 'anon_model' NOW (CHAIN 2).
#> Chain 2:
#> Chain 2: Gradient evaluation took 3e-06 seconds
#> Chain 2: 1000 transitions using 10 leapfrog steps per transition would take 0.03 seconds.
#> Chain 2: Adjust your expectations accordingly!
#> Chain 2:
#> Chain 2:
#> Chain 2: WARNING: There aren't enough warmup iterations to fit the
#> Chain 2: three stages of adaptation as currently configured.
#> Chain 2: Reducing each adaptation stage to 15%/75%/10% of
#> Chain 2: the given number of warmup iterations:
#> Chain 2: init_buffer = 7
#> Chain 2: adapt_window = 38
#> Chain 2: term_buffer = 5
#> Chain 2:
#> Chain 2: Iteration: 1 / 100 [ 1%] (Warmup)
#> Chain 2: Iteration: 10 / 100 [ 10%] (Warmup)
#> Chain 2: Iteration: 20 / 100 [ 20%] (Warmup)
#> Chain 2: Iteration: 30 / 100 [ 30%] (Warmup)
#> Chain 2: Iteration: 40 / 100 [ 40%] (Warmup)
#> Chain 2: Iteration: 50 / 100 [ 50%] (Warmup)
#> Chain 2: Iteration: 51 / 100 [ 51%] (Sampling)
#> Chain 2: Iteration: 60 / 100 [ 60%] (Sampling)
#> Chain 2: Iteration: 70 / 100 [ 70%] (Sampling)
#> Chain 2: Iteration: 80 / 100 [ 80%] (Sampling)
#> Chain 2: Iteration: 90 / 100 [ 90%] (Sampling)
#> Chain 2: Iteration: 100 / 100 [100%] (Sampling)
#> Chain 2:
#> Chain 2: Elapsed Time: 0 seconds (Warm-up)
#> Chain 2: 0 seconds (Sampling)
#> Chain 2: 0 seconds (Total)
#> Chain 2:
#>
#> SAMPLING FOR MODEL 'anon_model' NOW (CHAIN 3).
#> Chain 3:
#> Chain 3: Gradient evaluation took 2e-06 seconds
#> Chain 3: 1000 transitions using 10 leapfrog steps per transition would take 0.02 seconds.
#> Chain 3: Adjust your expectations accordingly!
#> Chain 3:
#> Chain 3:
#> Chain 3: WARNING: There aren't enough warmup iterations to fit the
#> Chain 3: three stages of adaptation as currently configured.
#> Chain 3: Reducing each adaptation stage to 15%/75%/10% of
#> Chain 3: the given number of warmup iterations:
#> Chain 3: init_buffer = 7
#> Chain 3: adapt_window = 38
#> Chain 3: term_buffer = 5
#> Chain 3:
#> Chain 3: Iteration: 1 / 100 [ 1%] (Warmup)
#> Chain 3: Iteration: 10 / 100 [ 10%] (Warmup)
#> Chain 3: Iteration: 20 / 100 [ 20%] (Warmup)
#> Chain 3: Iteration: 30 / 100 [ 30%] (Warmup)
#> Chain 3: Iteration: 40 / 100 [ 40%] (Warmup)
#> Chain 3: Iteration: 50 / 100 [ 50%] (Warmup)
#> Chain 3: Iteration: 51 / 100 [ 51%] (Sampling)
#> Chain 3: Iteration: 60 / 100 [ 60%] (Sampling)
#> Chain 3: Iteration: 70 / 100 [ 70%] (Sampling)
#> Chain 3: Iteration: 80 / 100 [ 80%] (Sampling)
#> Chain 3: Iteration: 90 / 100 [ 90%] (Sampling)
#> Chain 3: Iteration: 100 / 100 [100%] (Sampling)
#> Chain 3:
#> Chain 3: Elapsed Time: 0 seconds (Warm-up)
#> Chain 3: 0 seconds (Sampling)
#> Chain 3: 0 seconds (Total)
#> Chain 3:
#>
#> SAMPLING FOR MODEL 'anon_model' NOW (CHAIN 4).
#> Chain 4:
#> Chain 4: Gradient evaluation took 3e-06 seconds
#> Chain 4: 1000 transitions using 10 leapfrog steps per transition would take 0.03 seconds.
#> Chain 4: Adjust your expectations accordingly!
#> Chain 4:
#> Chain 4:
#> Chain 4: WARNING: There aren't enough warmup iterations to fit the
#> Chain 4: three stages of adaptation as currently configured.
#> Chain 4: Reducing each adaptation stage to 15%/75%/10% of
#> Chain 4: the given number of warmup iterations:
#> Chain 4: init_buffer = 7
#> Chain 4: adapt_window = 38
#> Chain 4: term_buffer = 5
#> Chain 4:
#> Chain 4: Iteration: 1 / 100 [ 1%] (Warmup)
#> Chain 4: Iteration: 10 / 100 [ 10%] (Warmup)
#> Chain 4: Iteration: 20 / 100 [ 20%] (Warmup)
#> Chain 4: Iteration: 30 / 100 [ 30%] (Warmup)
#> Chain 4: Iteration: 40 / 100 [ 40%] (Warmup)
#> Chain 4: Iteration: 50 / 100 [ 50%] (Warmup)
#> Chain 4: Iteration: 51 / 100 [ 51%] (Sampling)
#> Chain 4: Iteration: 60 / 100 [ 60%] (Sampling)
#> Chain 4: Iteration: 70 / 100 [ 70%] (Sampling)
#> Chain 4: Iteration: 80 / 100 [ 80%] (Sampling)
#> Chain 4: Iteration: 90 / 100 [ 90%] (Sampling)
#> Chain 4: Iteration: 100 / 100 [100%] (Sampling)
#> Chain 4:
#> Chain 4: Elapsed Time: 0 seconds (Warm-up)
#> Chain 4: 0 seconds (Sampling)
#> Chain 4: 0 seconds (Total)
#> Chain 4:
#> Warning: Bulk Effective Samples Size (ESS) is too low, indicating posterior means and medians may be unreliable.
#> Running the chains for more iterations may help. See
#> https://mc-stan.org/misc/warnings.html#bulk-ess
#> Warning: Tail Effective Samples Size (ESS) is too low, indicating posterior variances and tail quantiles may be unreliable.
#> Running the chains for more iterations may help. See
#> https://mc-stan.org/misc/warnings.html#tail-ess
Created on 2021-01-20 by the reprex package (v0.3.0)
Session info
devtools::session_info()
#> - Session info ---------------------------------------------------------------
#> setting value
#> version R version 4.0.2 (2020-06-22)
#> os Windows 7 x64 SP 1
#> system x86_64, mingw32
#> ui RTerm
#> language (EN)
#> collate Dutch_Belgium.1252
#> ctype Dutch_Belgium.1252
#> tz Europe/Paris
#> date 2021-01-20
#>
#> - Packages -------------------------------------------------------------------
#> ! package * version date lib source
#> assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.0.0)
#> callr 3.5.1 2020-10-13 [1] CRAN (R 4.0.3)
#> cli 2.2.0 2020-11-20 [1] CRAN (R 4.0.3)
#> codetools 0.2-16 2018-12-24 [2] CRAN (R 4.0.2)
#> colorspace 2.0-0 2020-11-11 [1] CRAN (R 4.0.3)
#> crayon 1.3.4 2017-09-16 [1] CRAN (R 4.0.0)
#> curl 4.3 2019-12-02 [1] CRAN (R 4.0.0)
#> desc 1.2.0 2018-05-01 [1] CRAN (R 4.0.0)
#> devtools 2.3.1 2020-07-21 [1] CRAN (R 4.0.2)
#> digest 0.6.27 2020-10-24 [1] CRAN (R 4.0.3)
#> dplyr 1.0.2 2020-08-18 [1] CRAN (R 4.0.2)
#> ellipsis 0.3.1 2020-05-15 [1] CRAN (R 4.0.2)
#> evaluate 0.14 2019-05-28 [1] CRAN (R 4.0.0)
#> fansi 0.4.1 2020-01-08 [1] CRAN (R 4.0.0)
#> fs 1.5.0 2020-07-31 [1] CRAN (R 4.0.2)
#> generics 0.1.0 2020-10-31 [1] CRAN (R 4.0.3)
#> ggplot2 3.3.2 2020-06-19 [1] CRAN (R 4.0.2)
#> glue 1.4.2 2020-08-27 [1] CRAN (R 4.0.2)
#> gridExtra 2.3 2017-09-09 [1] CRAN (R 4.0.0)
#> gtable 0.3.0 2019-03-25 [1] CRAN (R 4.0.0)
#> highr 0.8 2019-03-20 [1] CRAN (R 4.0.0)
#> htmltools 0.5.0 2020-06-16 [1] CRAN (R 4.0.2)
#> inline 0.3.17 2020-12-01 [1] CRAN (R 4.0.3)
#> jsonlite 1.7.2 2020-12-09 [1] CRAN (R 4.0.3)
#> knitr 1.30 2020-09-22 [1] CRAN (R 4.0.3)
#> lifecycle 0.2.0 2020-03-06 [1] CRAN (R 4.0.0)
#> loo 2.4.1 2020-12-09 [1] CRAN (R 4.0.3)
#> magrittr 2.0.1 2020-11-17 [1] CRAN (R 4.0.3)
#> matrixStats 0.56.0 2020-03-13 [1] CRAN (R 4.0.0)
#> memoise 1.1.0 2017-04-21 [1] CRAN (R 4.0.0)
#> munsell 0.5.0 2018-06-12 [1] CRAN (R 4.0.0)
#> pillar 1.4.7 2020-11-20 [1] CRAN (R 4.0.3)
#> pkgbuild 1.2.0 2020-12-15 [1] CRAN (R 4.0.2)
#> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.0.0)
#> pkgload 1.1.0 2020-05-29 [1] CRAN (R 4.0.2)
#> prettyunits 1.1.1 2020-01-24 [1] CRAN (R 4.0.0)
#> processx 3.4.5 2020-11-30 [1] CRAN (R 4.0.3)
#> ps 1.5.0 2020-12-05 [1] CRAN (R 4.0.3)
#> purrr 0.3.4 2020-04-17 [1] CRAN (R 4.0.0)
#> R6 2.5.0 2020-10-28 [1] CRAN (R 4.0.3)
#> Rcpp 1.0.5 2020-07-06 [1] CRAN (R 4.0.2)
#> D RcppParallel 5.0.2 2020-06-24 [1] CRAN (R 4.0.3)
#> remotes 2.2.0 2020-07-21 [1] CRAN (R 4.0.2)
#> rlang 0.4.9 2020-11-26 [1] CRAN (R 4.0.3)
#> rmarkdown 2.4 2020-09-30 [1] CRAN (R 4.0.3)
#> rprojroot 2.0.2 2020-11-15 [1] CRAN (R 4.0.3)
#> rstan * 2.26.0.9000 2021-01-20 [1] local
#> scales 1.1.1 2020-05-11 [1] CRAN (R 4.0.2)
#> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 4.0.0)
#> StanHeaders * 2.26.0.9000 2021-01-20 [1] local
#> stringi 1.5.3 2020-09-09 [1] CRAN (R 4.0.2)
#> stringr 1.4.0 2019-02-10 [1] CRAN (R 4.0.0)
#> testthat 3.0.0 2020-10-31 [1] CRAN (R 4.0.3)
#> tibble 3.0.4 2020-10-12 [1] CRAN (R 4.0.3)
#> tidyselect 1.1.0 2020-05-11 [1] CRAN (R 4.0.2)
#> usethis 1.6.1 2020-04-29 [1] CRAN (R 4.0.0)
#> V8 3.4.0 2020-11-04 [1] CRAN (R 4.0.3)
#> vctrs 0.3.5 2020-11-17 [1] CRAN (R 4.0.3)
#> withr 2.4.0 2021-01-16 [1] CRAN (R 4.0.2)
#> xfun 0.19 2020-10-30 [1] CRAN (R 4.0.3)
#> yaml 2.2.1 2020-02-01 [1] CRAN (R 4.0.0)
#>
#> [1] C:/R/Library
#> [2] C:/Program Files/R/R-4.0.2/library
#>
#> D -- DLL MD5 mismatch, broken installation.
library(rstan) st <- ' functions{ real plus(real a){ return a + 1; } } ' testcpp <- stanc(model_code = st)
Ok. I'll look into it. It's likely b/c I added a preprocessor. Can you try: testcpp <- stanc(file = "", model_code = st) for now?
That works and everything compiles and runs, and looks like normal, but the log probability / parameter values are all wrong. Will take me some time to see why.
That works and everything compiles and runs, and looks like normal, but the log probability / parameter values are all wrong. Will take me some time to see why.
In the current version I'm testing, it complains about plus in your code. Need to rename it.
Yes, plus is a Stan function as well and user defined functions can not shadow Stan function names.
That was just a cheap mwe, not my actual test problem ;) but sure. I've narrowed down the problem to the gradient calculation of some limited set of functions, but the functions on their own seem ok... will try some more.
Additionally, I posted a (edit) very silly issue and closed here: https://github.com/stan-dev/stanc3/issues/801 but maybe a more informative error could be considered sometime :)
One further thing that could be done for testing is to use the revdepcheck package from RStudio et. al. to check whether this new version breaks any of the existing CRAN packages with dependencies on {rstan}. Unfortunately I don't think I have the time nor computing power to do it myself at the moment though.
@jeffreypullin I did that (in my environment: Linux + Intel compilers + R-devel) and I'm using this version in production with almost all of its dependencies. I think we've two tasks:
- Maintainers review the code to make sure they're happy with the changes for future development.
- Users and/or devs test it in their use cases to catch any hidden errors (e.g., the
stancfileissue that I've fixed).
The good thing is that this is now following Stan development and the package deployment has been automated. This can ensure that rstan development is coupled with Stan.
Oh wow, sorry @hsbadr I didn't realize you'd already done that! I completely agree with 1 and 2.
The problem is the rstan tests arent really that rich. For example this case with the model code should have definitely been caught by at least one unit test. Expanding unit tests is definitely something we need to do in order to simplify the release cycle.
Ok. I spent the morning diagnosing the 'incorrect gradient under certain conditions' thing, and it magically fixed itself :/ No idea how / why, maybe some remnant left in memory? Anyway, tests are passing, seems fine, speed seems nicely bumped up!
One thing: in rstan 2.21, I got this:
Variable "JAX" does not exist.
error in 'model_ctsmgen' at line 1055, column 46
-------------------------------------------------
1053:
1054: asymDIFFUSION[ derrind, derrind ] =
1055: to_matrix( (add_diag( -sqkron_prod(JAX[ derrind, derrind ], JAX[ derrind, derrind ]),
but this parses in rstan 2.26 and then gives a compile error.
@cdriveraus @rok-cesnovar I think it's more of stanc --allow-undefined option. The variable JAX seems to be undefined and in one version it's allowed in stanc and failed in compilation and in the other one stanc rejected` it. You may need to provide the headers that define it.
Not sure I follow. It was just a miscapitalisation (should have been JAx) .
Hm, I get
Semantic error in 'string', line 917, column 43 to column 46: Identifier 'JAX' not in scope.
Hm, I get
Semantic error in 'string', line 917, column 43 to column 46: Identifier 'JAX' not in scope.
I think this is likely a valid error and may depend on the compiler version and flags. Does it run with the old version of rsran? If not, then, it should be in a separate issue, not related to this one.
yes that is a valid error. However, when I use stanc("", model_code=x) , it doesn't give the error, which is the problem.
yes that is a valid error. However, when I use stanc("", model_code=x) , it doesn't give the error, which is the problem.
It just allows undefined variables, not really an issue with rstan as it's a valid error and stanc3 didn't complain. So, @rok-cesnovar, is there an option in stanc3 to check undefined variables and how to pass it to the JS?
Hm, I am not sure why would it allow undefined variables? If I place that model here: https://rok-cesnovar.github.io/stanc3js-demo/index.html I get the error. And this link uses the exact same javascript file. So the issue might be rstan does not catch the error? Will check.
if I use stanc(file=x) instead of the model_code approach, it gives the error, so I don't think this is intended (or at least, consistent!) behaviour.
@cdriveraus @rok-cesnovar It does work as expected. Have you installed the latest rstan 2.26 binaries?
> library(rstan)
Loading required package: StanHeaders
rstan (Version 2.26.0.9000, GitRev: 2e1f913d3ca3)
For execution on a local, multicore CPU with excess RAM we recommend calling
options(mc.cores = parallel::detectCores()).
To avoid recompilation of unchanged Stan programs, we recommend calling
rstan_options(auto_write = TRUE)
> cppcode <- stanc(file = "ctsm.stan")
Info: Found int division at 'string', line 57, column 10 to column 17:
(d * d - d) / 2
Values will be rounded towards zero. If rounding is not desired you can write
the division as
(d * d - d) / 2.0
0
Semantic error in 'string', line 867, column 43 to column 46:
Identifier 'JAX' not in scope.
> cppcode$status
[1] FALSE
Maybe it's confusing because 'giving an error' is actually the desired behaviour. yes, stanc(file=x) works correctly. stanc(file="",model_code=x) does not return an error, which seems incorrect.
Maybe it's confusing because 'giving an error' is actually the desired behaviour. yes, stanc(file=x) works correctly. stanc(**file="",**model_code=x) does not return an error, which seems incorrect.
That's a hack to test the older version; don't use it 😄
And, yes, I can convert this to Error and catch the hack too.
@cdriveraus @rok-cesnovar I've pushed a few commits to catch those cases and stop on stanc3 errors. Give it a try after it builds the binaries and let me know if it addresses the problem.