testing with testthat for expect_that errors in packages does not work

[maxheld83]

disclaimer: this might be duplicate to #76 not sure

I like to use checkmate's and testthat's expect_* functions for input validation inside packages.

Additionally, I then sometimes like to check for expected error messages with a testthat test.

That doesn't seem to work.

For example:

test_that(desc = "errors out on real names that are also fake names",
          code = {
  file <- data.frame(real_names = c("Hillary", "Barack"),
                     fake_names = c("Hillary", "John"),
                     stringsAsFactors = FALSE)
  write.csv(x = file, file = "test.temp.csv", row.names = FALSE)
  expect_error(object = anonymize(real_names = c("Hillary", "Barack"),
                                  lookup_file = "test.temp.csv"))
})

gives:

Error: Test failed: 'errors out on real names that are also fake names'
* i %in% file$fake_names isn't false.
Hillary is a real name but also used as a fake name.
* anonymize(real_names = c("Hillary", "Barack"), lookup_file = "test.temp.csv") did not throw an error.

even though I expected an error, that is what I wanted.

My hunch is that this problem arises, because I am using expect_true inside anonymize() to throw the very error I am here testing. That kind of nested stuff seems to throw off testthat.

Did I get this right?

Anything I can do about this?

Ps.: apologies for the not fully REPEX; let me know if you need one, I thought the problem might be clear as is.

[mllg]

My MWE:

context("nested expect")

foo = function(x) {
  expect_true(x)
  stop("myerror")
}

test_that("nested expect", {
  expect_error(foo("a"), "myerror")
})

Seems like a general testthat issue. You could try to open an issue for testthat to get this fixed.

If the anonymize function is part of the package, it should raise an exception and not rely on testthat. Otherwise, if this is a function for unit tests only, you could try to either stick to only expects or only assertions/stops. Note that you can also easily create your own expectations using ?makeExpectation or ?makeExpectationFunction.