Unit tests (using pFunit)

[dorchard]

I have started putting in the infrastructure for us to write some unit tests using pFunit.

Create simple unit tests for main functions

• [ ] Make the cmake portable; I fought a bit, but I don't know enough cmake. We need to be able to link the include directories for ftorch. At the moment the library and include paths need to be specified.
• [ ] Test torch_tensor_from_blob (WIP)
• [ ] Test torch_tensor_from_array

Other things using torch_module_forward could be better with an integration test since it will need a lot of setup code anyway.

Closing this would fix #4

[dorchard]

tests/test_ftorch.pf gives a start of a unit test for torch_tensor_from_blob

[jatkinson1000]

Has this gone anywhere, or has it stagnated?

As far as I see it tests are the only thing preventing us from making a JOSS submission, and with at least 3 research projects using FTorch that are likely to publish soon this is now a matter of urgency.

For something minimal I was thinking we could have some CMake Test to build and run the SimpleNet example for a few different inputs and check the outputs are correct as an integration test.

[dorchard]

There's a start here, but it got a bit stalled as I could use some help with the cmake. Perhaps someone is able to work with me on this? @TomMelt ? Currently has an attempt at using CMake test and does a simple probe of the tensor conversion functions. I agree about running SimpleNet for some integration tests.

[jatkinson1000]

Great, thanks @dorchard Proposal from the morning is to briefly pause on this in favour of integration tests in #115 so we can make a JOSS submission, then return to this. We can certainly take a look and help with cmake, and the integration tests will use cmake so you might find those useful.

[TomMelt]

Hi @dorchard I'm more than happy to help with the CMake side of things. Let me know when is convenient.

[joewallwork]

Perhaps this PR could just set up the infrastructure and add tests for the tensor constructors and we could address writing unit tests for the rest of FTorch in a follow-up PR?

[joewallwork]

Interestingly, I needed to set the integer kind to get these tests working on my Ubuntu laptop, too (see https://github.com/Cambridge-ICCS/FTorch/pull/76/commits/c30bc218440d641e52ead288abbf4dc5e13c9a93). I'll update this to follow @TomMelt's 32-bit int approach once #137 is merged.

[joewallwork]

[Rebased on top of main to bring in linting changes]

[joewallwork]

Thanks for the tips @TomMelt - with a few tweaks the tests now pass on Ubuntu. There's a build fail in the Windows CI, although I didn't edit the Windows CI workflow or the part of the CMakeLists that it errors at. Do you have any thoughts on what's gone wrong? https://github.com/Cambridge-ICCS/FTorch/actions/runs/12351579042/job/34466833553?pr=76

[joewallwork]

Thanks for the tips @TomMelt - with a few tweaks the tests now pass on Ubuntu. There's a build fail in the Windows CI, although I didn't edit the Windows CI workflow or the part of the CMakeLists that it errors at. Do you have any thoughts on what's gone wrong? https://github.com/Cambridge-ICCS/FTorch/actions/runs/12351579042/job/34466833553?pr=76

Huh, I'm getting a similar error on a PR that doesn't change any of the build system or source code (#206): https://github.com/Cambridge-ICCS/FTorch/actions/runs/12352023942/job/34468182561?pr=206

[joewallwork]

Merged in the Windows fix from #207 in https://github.com/Cambridge-ICCS/FTorch/pull/76/commits/23ce977bfa9068797a4f9be0ea183f6290e2f54a. Turned off unit testing for Windows in https://github.com/Cambridge-ICCS/FTorch/pull/76/commits/f1e9add826c8b02f87e6d53f1380d7d7b55f3521.

[joewallwork]

@dorchard I can't add you as a reviewer because you authored the initial commits, but any comments you have would be appreciated.

[joewallwork]

Went through this in the meeting today; up-to-date with main and ready for review.

[dorchard]

I'm having issues building the unit tests, e.g.:

[ 37%] Building Fortran object test/unit/CMakeFiles/test_constructors.dir/test_constructors.F90.o
/Users/dorchard/Documents/iccs/FTorch/src/build/test/unit/test_constructors.F90:10:7:

   10 |   use pFUnit
      |       1

I am using pfunit-4.10 whose mod files seem to be named funit.mod instead (changing use pFunit to use funit in test_constructors.f90 makes things build...) Has there been a breaking interface change for pfunit? Is it because I built pfUnit skipping the OpenMP and MPI builds I wonder? I can't seem to identify whether this actually changes the mod file though. Maybe I am missing something obvious in the build setup.

[joewallwork]

I'm having issues building the unit tests, e.g.:

[ 37%] Building Fortran object test/unit/CMakeFiles/test_constructors.dir/test_constructors.F90.o
/Users/dorchard/Documents/iccs/FTorch/src/build/test/unit/test_constructors.F90:10:7:

   10 |   use pFUnit
      |       1

I am using pfunit-4.10 whose mod files seem to be named funit.mod instead (changing use pFunit to use funit in test_constructors.f90 makes things build...) Has there been a breaking interface change for pfunit? Is it because I built pfUnit skipping the OpenMP and MPI builds I wonder? I can't seem to identify whether this actually changes the mod file though. Maybe I am missing something obvious in the build setup.

Yes, you need to install with MPI to get pFUnit (p for parallel). However, I realised that isn't actually used at present so switched to use FUnit in d52c389 in the current tests, so that should work for you.

[joewallwork]

Note that I've opened #216 to fix a related issue with torch_tensor_from_array. It will merge into this PR.

[joewallwork]

I think this is a good starting point for our unit tests. I had one suggestion. Having a 3D test would also be good I think, at least on the torch_tensor_from_array side.

Good suggestion. I added this in 4dad25e945000be5ae005b4f9f56608d8686bdc8, which required extending the assert_allclose test util to cover the 3D case.

[dorchard]

Looks good to go for me!

[joewallwork]

Thanks again everyone! Merging now :rocket: