Add end to end test devoted to ctest

[hippo91]

This changes test system

The following changes are proposed:

This PR is a proposal to add end to end test devoted to ctest part of the extension.

It adds a directory named single-root-ctest in the end-to-end-tests directory.

It also changes slightly the source directory in order:

• to avoid early return in the refreshTests method (ctest.ts file) when in testing mode;
• to change the return code of the ctest method (cmakeProject.ts) so that successfull test return 0 and failure returns -1.

The purpose of this change

When doing change to the ctest part of the extension, i did not feel very comfortable not being backed up by tests. This PR tries to circumvent this.

[gcampbell-msft]

@hippo91 I apologize for taking so long to start reviewing these, we appreciate your patience.

We GREATLY appreciate your contribution to our engineering system, so we'd love to work with you and iterate on this PR to make these testing contributions,

A couple of things:

1. These tests should be integrated into our PR build pipelines so that we can test them in each PR
2. To do that, they should be set up to be cross-platform so that they can be run on each platform, since ctest is available on all three OS's. This will require modifications to the presets to define things like the CMAKE_C_COMPILER, CMAKE_CXX_COMPILER, and possibly using different generators on each OS.
3. Currently, are you able to get these to work locally on Windows? I notice that I hit errors when trying to open and run in Windows.

Again, we greatly appreciate this contribution and hope to iterate on the PR with you so that we can improve our engineering system.

Thanks! Let us know if you have any questions.

[hippo91]

@gcampbell-msft i integrated these tests in the PR build pipelines.

I configured C and CXX compilers for linux and windows systems. I do not have any windows os at home so i'm not able to test that. Neither can i on Mac OS. I was counting on the CI to check that.

[hippo91]

@gcampbell-msft i tried to debug the failing CI tests but was unable to understand what was failing in the build step.

I suppose this is the build of the C++ test project but cannot confirm. For now i try to run the CI locally using act. If you have any idea or hint about how i can debug the CI i'll be happy to try them.

[gcampbell-msft]

@hippo91 Update, it looks like the builds are failing tests due to an unexpected temp folder, possibly due to differences in OS's. Could you investigate?

OR, please let us know if you'd like help with this PR, and we can aim to help out on this PR for milestone 1.21.

[hippo91]

@gcampbell-msft i am trying to investigate but i do not know how i can have access to build output or log. Have you got any idea? Another approach that i'am trying is to run the CI locally thanks to act. Maybe is there another possibility?

[gcampbell-msft]

@hippo91 Are you not able to view the build log if you click on the github links?

Our CI pipelines run publicly in github. Here is a link to the Windows one, for example, https://github.com/microsoft/vscode-cmake-tools/actions/runs/14145701868/job/40095177801?pr=4186

[hippo91]

@gcampbell-msft i am able to see the build log but for linux the only interesting thing is Warning: ls] 2025-04-07T11:17:46.863Z [warning] [proc] The command: /home/runner/work/_temp/-1604695791/cmake-3.18.3-Linux-x86_64/bin/cmake --build /home/runner/work/vscode-cmake-tools/vscode-cmake-tools/test/end-to-end-tests/single-root-ctest/project-folder/build -- exited with code: 1 but i have no access to the inner log of this command.

It seems like the cmake command does not receive the correct config preset. It is really weird because on my laptop (running debian) using act, the CI is successful.

[gcampbell-msft]

@hippo91 I'm also interested in the failure for the Windows build. However, I also can't see any more information. I'll do my best to make time to investigate this sometime in the next week or so. If I don't get to it, we do hope to take this in our 1.21 milestone, which we hope to release in the next month or two. Thanks for your patience.

[hippo91]

@gcampbell-msft the problem with windows is due to the fact i dumbly hardcoded path to /tmp which of course does not exist on windows. I should be able to fix this. However the only way for me to check windows execution is through the CI.

And for Linux in the CI for now i have no clue.

[hippo91]

@gcampbell-msft i think i fixed the windows problem. Can you relaunch the CI please?

[hippo91]

@gcampbell-msft surprisingly linux ci seems to be ok.

I removed an unused CMAKE variable for windows. It was generating what could be an error.

[hippo91]

@gcampbell-msft the linux part of the CI is ok but i'm struggling with the windows one (the macos one is failing in an unrelated test). If you could have a look it would be nice. Please tell me if i can do anything to help.

[gcampbell-msft]

@hippo91 I updated to main and re-ran, it looks like the build for macos is failing even before running the tests, unfortunately I won't be able to investigate today, but we still hope to take this PR and we really appreciate your hard work!