testthat
testthat copied to clipboard
r-lib
Test failures in version 3.2.1.1
Hi, I tried to upgrade the Debian packaged testthat from version 3.2.1 to 3.2.1.1. We are running a test script in our CI which boils down to
cd tests
LC_ALL=C.UTF-8 R --no-save < testthat.R
inside the source tree (and when doing so I get the very same result). The CI contains a full build log which results in
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test-mock2.R:62:1'): can mock bindings in another package ───────────
Error in `call_type(call)`: corrupt language object
── Failure ('test-reporter-list.R:54:3'): exercise ListReporter ────────────────
expectation_type(res[[4]]$results[[1]]) (`actual`) not identical to "failure" (`expected`).
`actual`: "warning"
`expected`: "failure"
── Failure ('test-reporter-list.R:57:3'): exercise ListReporter ────────────────
df$error (`actual`) not equal to c(FALSE, FALSE, FALSE, FALSE, TRUE) (`expected`).
`actual[2:5]`: FALSE FALSE FALSE FALSE
`expected[2:5]`: FALSE FALSE FALSE TRUE
── Failure ('test-reporter-debug.R:51:3'): debug reporter is not called for successes ──
get_frame_from_debug_reporter(2, success_fun) is not NULL
`actual` is an environment
`expected` is NULL
── Failure ('test-srcrefs.R:19:3'): line numbers captured for expectations and warnings ──
`lines` (`actual`) not equal to c(2, 3) (`expected`).
`actual`: 2 2 3
`expected`: 2 3
── Failure ('test-srcrefs.R:32:3'): line numbers captured when called indirectly ──
`lines` (`actual`) not equal to 4 (`expected`).
`actual`: 4 4
`expected`: 4
── Failure ('test-srcrefs.R:42:3'): line numbers captured when called indirectly ──
`lines` (`actual`) not equal to 5 (`expected`).
`actual`: 5 5
`expected`: 5
── Failure ('test-srcrefs.R:51:3'): line numbers captured inside a loop ────────
`lines` (`actual`) not equal to rep(2, 4) (`expected`).
`actual[2:8]`: 2 2 2 2 2 2 2
`expected[2:4]`: 2 2 2
── Error ('test-test-that.R:55:5'): infinite recursion is captured ─────────────
<expressionStackOverflowError/stackOverflowError/error/condition>
Error: evaluation nested too deeply: infinite recursion / options(expressions=)?
Backtrace:
▆
1. ├─testthat::with_reporter(...) at test-test-that.R:54:3
2. │ └─base::tryCatch(...)
3. │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
4. │ └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
5. │ └─base (local) doTryCatch(return(expr), name, parentenv, handler)
6. ├─withr::with_options(...) at test-test-that.R:55:5
7. │ └─base::force(code)
8. ├─testthat::test_that(...)
9. │ └─testthat::local_test_context()
10. │ └─testthat::local_reproducible_output(.env = .env)
11. │ └─withr::local_language(lang, .local_envir = .env)
12. │ └─withr:::check_language_envvar("LC_ALL")
13. │ └─base::warning(...)
14. ├─base::.signalSimpleWarning(...)
15. │ └─base::withRestarts(...)
16. │ └─base (local) withOneRestart(expr, restarts[[1L]])
17. │ └─base (local) doWithOneRestart(return(expr), restart)
18. ├─testthat (local) `<fn>`(`<smplWrnn>`)
19. │ └─rlang::cnd_entrace(e)
20. │ └─rlang::trace_back(top = top, bottom = bottom)
21. │ └─rlang:::map(calls, call_zap_inline)
22. │ └─base::lapply(.x, .f, ...)
23. │ └─rlang (local) FUN(X[[i]], ...)
24. └─rlang:::call_type_sum(x)
25. ├─rlang::sym(sprintf("<%s>", rlang_type_sum(x)))
26. │ └─rlang::is_symbol(x)
27. ├─base::sprintf("<%s>", rlang_type_sum(x))
28. └─rlang:::rlang_type_sum(x)
29. └─rlang::is_installed("pillar")
30. └─rlang:::detect_installed(info)
31. ├─rlang:::list_c(...)
32. │ └─rlang::inject(c(!!!x))
33. │ └─rlang::enexpr(expr)
34. └─rlang:::pmap(...)
35. └─rlang:::.rlang_purrr_args_recycle(.l)
36. └─rlang:::map_int(args, length)
37. └─rlang:::.rlang_purrr_map_mold(.x, .f, integer(1), ...)
38. └─base::vapply(.x, .f, .mold, ..., USE.NAMES = FALSE)
── Failure ('test-test-that.R:102:3'): no braces required in testthat 2e ───────
`test_that("", expect_true(TRUE))` generated warnings:
* Changing language has no effect when envvar LC_ALL='C.UTF-8'
[ FAIL 10 | WARN 1412 | SKIP 122 | PASS 819 ]
Please let me know if you need further information to track down the issue. In case you can't reproduce the issue the Debian Med team policy contains a hint to a docker file which could be useful to reproduce the problem on a Debian sid system. Kind regards, Andreas.
@tillea
Just for curiosity what are the changes between versions 3.2.1.1 and 3.2.1?
As I can't find ChangeLog or News file related I did a simple diff between versions:
diff -ruN 3.2.1/DESCRIPTION 3.2.1.1/DESCRIPTION
--- 3.2.1/DESCRIPTION 2023-12-02 11:50:05.000000000 +0000
+++ 3.2.1.1/DESCRIPTION 2024-04-14 04:25:11.000000000 +0100
@@ -1,6 +1,6 @@
Package: testthat
Title: Unit Testing for R
-Version: 3.2.1
+Version: 3.2.1.1
Authors@R: c(
person("Hadley", "Wickham", , "[email protected]", role = c("aut", "cre")),
person("Posit Software, PBC", role = c("cph", "fnd")),
@@ -31,10 +31,10 @@
Encoding: UTF-8
RoxygenNote: 7.2.3
NeedsCompilation: yes
-Packaged: 2023-11-30 22:07:21 UTC; hadleywickham
+Packaged: 2024-04-14 03:24:27 UTC; ripley
Author: Hadley Wickham [aut, cre],
Posit Software, PBC [cph, fnd],
R Core team [ctb] (Implementation of utils::recover())
Maintainer: Hadley Wickham <[email protected]>
Repository: CRAN
-Date/Publication: 2023-12-02 11:50:05 UTC
+Date/Publication: 2024-04-14 05:24:52
diff -ruN 3.2.1/MD5 3.2.1.1/MD5
--- 3.2.1/MD5 2023-12-02 11:50:05.000000000 +0000
+++ 3.2.1.1/MD5 2024-04-14 04:25:11.000000000 +0100
@@ -1,4 +1,4 @@
-7a02630a3e38f7521fb5c44a3f697a8a *DESCRIPTION
+7ad03c426643c38668b5fe0673a9a970 *DESCRIPTION
d0b909bd833cd58ea0430eda333b6447 *LICENSE
98b8d9fa7a3c0f90c19158aa47ec85f1 *NAMESPACE
b682a5016b2f64f2b8704f47aa54c3eb *NEWS.md
@@ -113,7 +113,7 @@
1ece0759f60e98193d53545648d19146 *inst/examples/test-failure.R
522cf134c79624117d579f2c4d18f42b *inst/examples/test-success.R
543392950ccc39ed50ebf26c75de0910 *inst/include/testthat.h
-5058228172d50a29a234414c78b25399 *inst/include/testthat/testthat.h
+05f098fedb1fe85ee85de806f711779f *inst/include/testthat/testthat.h
b5816f117a5460c67e3e08b1c23dacbc *inst/include/testthat/vendor/catch.h
f481e7613929d4968a4a4c73031f7967 *inst/resources/catch-routine-registration.R
f35a879e88834e75aee99fc9817547a0 *inst/resources/test-cpp.R
diff -ruN 3.2.1/inst/include/testthat/testthat.h 3.2.1.1/inst/include/testthat/testthat.h
--- 3.2.1/inst/include/testthat/testthat.h 2023-11-29 18:00:12.000000000 +0000
+++ 3.2.1.1/inst/include/testthat/testthat.h 2024-04-14 04:23:53.000000000 +0100
@@ -172,7 +172,7 @@
extern "C" SEXP run_testthat_tests(SEXP use_xml_sxp) {
bool use_xml = LOGICAL(use_xml_sxp)[0];
bool success = testthat::run_tests(use_xml);
- return ScalarLogical(success);
+ return Rf_ScalarLogical(success);
}
# endif // TESTTHAT_TEST_RUNNER
@@ -198,7 +198,7 @@
# include <R.h>
# include <Rinternals.h>
extern "C" SEXP run_testthat_tests() {
- return ScalarLogical(true);
+ return Rf_ScalarLogical(true);
}
# endif // TESTTHAT_TEST_RUNNER
And there is changes in testthat.h that maybe causing the error.
I have yet not updated FreeBSD port, but soon I will test it too and share results.
Thanks
@tillea
I found that commit at https://github.com/r-lib/testthat/commit/93b88dbda3dd048e74dd2bce4623c9436d2013ff I will test it today
That release was performed by BDR, not by us, but I find it hard to believe that it's responsible for your test failures.
