Test failures on r-devel due to changes to all.equal() and testthat edition 3

[jgabry]

trafficstars

With r-devel all.equal(), which is used by expect_equal() in testthat edition 2, also compares environments so

expect_equal(ggplot_object_1, ggplot_object_2)

now fails when previously it would succeed. We use this in multiple places to test that certain function calls that should produce the same object actually do, for example:

https://github.com/stan-dev/bayesplot/blob/cb1b403c4fd46cbcc28564970cb88c35486001fa/tests/testthat/test-pp_check.R#L9-L12

and

https://github.com/stan-dev/bayesplot/blob/cb1b403c4fd46cbcc28564970cb88c35486001fa/tests/testthat/test-ppc-scatterplots.R#L17-L22

At the same time, if we switch to the new testthat edition 3, then expect_equal() will calls waldo::compare() instead of all.equal() and waldo::compare() doesn't provide a way to turn off environment checking (it does but only in a few cases like comparing functions that aren't sufficient to make it work with ggplot objects). So that isn't an alternative.

We can possibly just use vdiffr instead of expect_equal() but vdiffr doesn't seem to provide a way to compare two ggplot objects to the same saved SVG file as far as I can tell.

[tjmahr]

waldo::compare() has two parameters that mention environments. Does that help?

We could build the plots and compare them. That would add plot building to the test timing.

[jgabry]

waldo::compare() has two parameters that mention environments. Does that help?

Those don’t help unfortunately. I found this open waldo issue https://github.com/r-lib/waldo/issues/56 and I added a few comments. But it seems like it’s not intended to work with ggplot objects.

We could build the plots and compare them. That would add plot building to the test timing.

Yeah we could try that.

[TimTaylor]

Does the example here help https://testthat.r-lib.org/reference/expect_snapshot_file.html. I'm currently using it to compare to svg plots.

[jgabry]

Thanks for the suggestion! Yeah that does look like it might work for our situation.

[jgabry]

I managed to work around this for now by adding check.environment=FALSE to the expect_equal() calls and continuing to use testthat edition 2. When we switch to edition 3 we'll need to change our approach, perhaps using the suggestion from @tjtnew to use expect_snapshot_file().

[jgabry]

CRAN wants a release soon that fixes the failing tests on R-devel. That's why I went with the workaround instead of testthat edition 3 and expect_snapshot_file().