Vignette on testing challenging functions

[maelle]

At the moment the only reference to mockery is via the deprecation message of former mocking functions.

Could there be a link to mockery maybe from the README?

[hadley]

README feels too prominent. Mocking is a pretty advanced topic. Maybe it could be part of a new vignette on "testing tough functions" or similar?

[maelle]

Sounds like a cool vignette topic. Aren't test fixtures part of "testing tough functions"?

[hadley]

Hmmmm, maybe related, but distinct? I was thinking more of a vignette that would cover functions that involve randomness, interaction, external data, and (maybe) plots.

[maelle]

https://blog.r-hub.io/2019/10/29/mocking/ was relatively popular (well for an R-hub post :slightly_smiling_face: ) so I think a vignette about such topics would be awesome.

[nbenn]

Unhappy with limitations presented by both 'mockery' and 'mockr' (the deal-breaker for me had to do with S3 dispatch; r-lib/mockery#43, krlmlr/mockr#9), I started working on my own solution: 'mockthat', which has recently been accepted on CRAN. It is powered by utils::assignInNamespace() and as far as I can tell it does not suffer from any of the issues I encountered with the 'mockery'/'mockr' approaches.

It combines ease of use from the (now-deprecated) 'testthat' approach with a a feature from 'mockery' which I found useful: you can stub out a function with a closure that keeps track of how the function in question was called. Unlike with the 'mockr' approach, you can stub out functions from other packages and unlike 'mockery' you do not have to precisely specify where the stubbing has to occur. Also in addition to with_mock(), a local_mock() function is available from 'mockthat' (powered by withr::defer()).

I'm happy to elaborate if this is of interest to anyone and I'm happy to contribute to such a proposed vignette as I have recently spent a bit of time on the topic. Also it would of course be great for visibility of my package if it was mentioned somewhere in a 'testthat' vignette.

[krlmlr]

Nice job!

I'll be working on integrating mockr with mockery, so that we can retire mockr and keep the old functionality for now -- https://github.com/nbenn/mockthat/issues/2 seems a bit of a risk here. I like the idea of adding a link to mockthat as an alternative. There's also stubthat which imports mockr.

CC @jimhester.

[maelle]

Regarding plots, some guidance on testthat visual snapshots vs vdiffr (or how the two tools complement each other) would be nice.

[hadley]

• Use of expect_snapshot() for length-y text output
• Plus use of mocking and/or transform to eliminate spurious differences

[hadley]

I made this list for a recent workshop on testing:

• Output affected by RNG — withr::local_seed()
• Output affected by other external state — withr::local_options() / withr::local_envvar()
• Graphical output — https://vdiffr.r-lib.org/
• Errors & other user facing text — snapshots
• HTTP responses — httr2 mocking + httptest2
• Interactivity — mocking

Are there any other major classes of challenge that you can think of?

[maelle]

Regarding interactivity (and other examples like "code that should be run if there is no internet connection"), maybe also mention escape hatches? https://blog.r-hub.io/2023/01/23/code-switch-escape-hatch-test/

[hadley]

@maelle in that case (especially with local_mocked_bindings()), mocking feels like a clear win.