tinytest
tinytest copied to clipboard
markvanderloo
exporting test results as structured data for CI
Hi, thanks for tinytest. Glad to see that not everyone thinks testing in R is a solved problem!
I'm interested to see how to integrate/export test results in CI flows. Directly supporting different integrations (like the reporters concept in testthat) probably goes against the philosophy of keeping things lightweight.
Is there an easy way to export the test results in a structured text data format e.g. json?
From there it would be easy enough to convert to e.g. JUnit xml format in DIY tooling.
Hi there!
You can use as.data.frame() on the output (tinytest object), and that can be translated to JSON using e.g. the JSONLite package.
I've been meaning to look into standardized test results. If I can produce them with simple templating and 'sprintf', for example, we could add this to the package.
I'm not aware what are the most popular formats, not which formats are used by which CI service. But I can imagine we could support something that is not too complicated and used in popular services. Any pointers would be appreciated.
I also want to mention that last time I checked (which is a while ago), tinytest works fine together with covr.
-Mark
On Wed, Feb 9, 2022 at 5:48 PM Daan Seynaeve @.***> wrote:
Hi, thanks for tinytest. Glad to see that not everyone thinks testing in R is a solved problem!
I'm interested to see how to integrate/export test results in CI flows. Directly supporting different integrations (like the reporters concept in testthat) probably goes against the philosophy of keeping things lightweight. Is there an easy way to export the test results in a structured text data format e.g. json. From there it would be easy enough to convert to e.g. JUnit xml format.
— Reply to this email directly, view it on GitHub https://github.com/markvanderloo/tinytest/issues/98, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOWSARQPFLTSMN6EB7VNCTU2KLHJANCNFSM5N6BRC3A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you are subscribed to this thread.Message ID: @.***>
Hi Mark,
we've started looking into this a bit further, for now in the form of separate package: https://github.com/openanalytics/tinytest2JUnit but interested to see if it could be useful to integrate anything upstream in tinytest itself
Cool! I think that a separate package is actually a great idea. This would both keep things tiny for the user and still give the opportunity to integrate with JUnit for those who need it. I'd be happy to document this, for example in the package's README when this package hits CRAN
https://cran.r-project.org/web/packages/tinytest2JUnit is available now :)
Awesome! I advertised a little bit: https://twitter.com/markvdloo/status/1667069773119717376?s=20 and https://mathstodon.xyz/@markvanderloo/110513061667258127