test-drive icon indicating copy to clipboard operation
test-drive copied to clipboard

Integrate with `fpm`

Open certik opened this issue 4 years ago • 4 comments
trafficstars

The fpm new should use some testing framework and set things up, so depend on test-drive and add a simple test. Then you can easily extend it as needed.

We should allow (in the future) to use any testing framework in fpm, configurable with a command line option. But we can use test-drive as the default.

So we need test only dependencies, as in Cargo.

certik avatar Sep 10 '21 18:09 certik

We already have the functionality in fpm to use development (i.e. testing) dependencies with

[dev-dependencies]
test-drive.git = "https://github.com/fortran-lang/test-drive"

Which makes the dependencies available to all tests, but does not make it a transient dependency for dependent projects.

Alternatively we also can limit the scope to a single test target by using

[[test]]
name = "tester"
[test.dependencies]
test-drive.git = "https://github.com/fortran-lang/test-drive"

In fpm new we can setup the boilerplate code from the README of this project by default, which provides a fully working testing framework with support for multiple suites with different unit tests.

awvwgk avatar Sep 10 '21 19:09 awvwgk

Perfect. Yes, let's start the template with test-drive and see how it works.

Then we can make it configurable, @everythingfunctional can help with vegetables support.

certik avatar Sep 10 '21 19:09 certik

Hey all, I wonder what's the current status of this effort? Are there plans to support also some form of coverage report after running fpm test?

EDIT: sorry, just discovered about fpm test --flag coverage and I see that is in fact quite effective and easy to set up in a CI. Happy that the cherry is already there :)

~~I think this would be ideal for CI workflows, which would then be managed as easily as making a single call to fpm and then feeding the report to Codecov or similar tools. This is actually very easy 'n comfy with Matlab, in fact easier then setting up a test-suite manually on a local machine, and I believe also fairly easy with Python (I don't have direct personal experience, but basically every Python codebase has publicly measured coverage nowadays, something that I see quite a bit lacking in our ecosystem).~~

Given the overall amazing user experience that fpm brings to the table I think this could serve as the proverbial cherry-on-top. 🙃🍒

beddalumia avatar Sep 15 '22 04:09 beddalumia

Full disclosure: I just started a completely new (i.e. with no trace of legacy stuff inside) fortran project these days and took my chance to give a full trial to the fpm experience. I am so amazed that I cannot see myself touching a CMakeList file ever again. So thank you from the heart, keep going with the fantastic tooling!

beddalumia avatar Sep 15 '22 04:09 beddalumia