Unable to stop the execution of CTest tests

[vdilecce]

Brief Issue Summary

Once command "CMake: Run Tests" has been run, I found no way of stopping the tests execution.

Expected:

1. Run command "CMake: Run Tests"
2. Be able to stop running tests either through a "Cancel" button in the notification, or through a command "CMake: Cancel Run Tests" (similarly to command "CMake: Cancel Build").

Apparent Behavior:

1. Run command "CMake: Run Tests"
2. I only succeded in stopping the tests by closing Visual Studio Code

Platform and Versions

• OS/Compiler: CentOS 7.9 / GCC 9.2 (though it's probably a platform-independent issue)
• CMake Version: 3.20.5
• VSCode Version: 1.59.1
• CMake Tools Extension Version: 1.8.0

[bobbrow]

Thank you for the feature suggestion. We may not be able to get to this immediately, but will accept PR's from the community if an enterprising developer out there is interested in tackling this before we get to it.

[github-actions[bot]]

This issue is now marked as 'stale-old' due to there being no activity on it for the past 720 days. Unless the 'stale-old' label is removed or the issue is commented on, this will be remain open for at least 14 days and then it may be closed. If you would like to make this issue exempt from getting stale, please add the 'stale-exempt' label.

[MikeWeller]

This is still a frustrating issue I hit just the other day. Posting to keep this alive.

[gcampbell-msft]

@MikeWeller Thank you for notifying us. Marking this as stale-exempt to keep it alive. @xisui-MSFT FYI.

[tvatter]

Still going on and still super frustrating.

[gcampbell-msft]

@tvatter Thanks for following up and reminding us of this frustration. I've upgraded this bug from the backlog to On Deck, which is where we pull bugs from to solve for upcoming releases

[kh-huawei]

Commenting for visibility. This is super annoying, especially if the test suites include a benchmark and take several hours to run, and i hit the 'Run CTest' - button by accident.

[dgazzoni]

Please fix this.

I teach a class using test-driven development, and as expected every so often a student inadvertently writes code with an infinite loop. They question me how they're supposed to stop the test run, and I ashamedly tell them that they have to restart VS Code, to which they usually give me a look back like "are you serious?". It's clear it's leaving a bad impression about VS Code on them, and I'm not sure how many of them will be eager to adopt it on the future.

[gcampbell-msft]

@dgazzoni @kh-huawei @tvatter @MikeWeller There actually is already support for this, assuming that you have the cmake.ctest.testExplorerIntegrationEnabled setting enabled. If you do, then you can navigate to the Testing Pane and hit the stop button:

However, to make this more obvious, I think it may be helpful to also implement the test executors with vscode.window.withProgress as well.

What do you all think? Does the Testing pane solution work for your cases?

[dgazzoni]

I just tried it. I have the cmake.ctest.testExplorerIntegrationEnabled setting enabled, I purposefully added an infinite loop into my code, ran the tests, and tried to stop them exactly on that button you highlighted.

They didn't stop, and the executable is still running, taking up 100% CPU.

This is with CMake Tools v1.20.18 (pre-release).

[gcampbell-msft]

@dgazzoni Thanks for following up, I also just tested this and I see the behavior you're seeing. I'm moments away from opening a PR that fixes this behavior for you, I'll also attach a vsix to the PR so that you can test.

Thanks for your patience.

[gcampbell-msft]

@dgazzoni @MikeWeller @tvatter @kh-huawei I've created a PR here #4141 and dropped a vsix in the description for you to test.

You'll need to download the zip file, and NOT extract it, but simply change the extension to .vsix, and you'll be able to install it manually and test it that way

[dgazzoni]

I just tested the proposed change. It does stop the tests, but there is an issue when cmake.ctest.allowParallelJobs is enabled: in that case it doesn't kill the test processes (without that setting it does), leaving some CPU-hogging processes running.

So if it would be possible to also kill the test processes with that setting enabled, that would be wonderful.

[gcampbell-msft]

@dgazzoni Thanks for testing that. I thought that I tested and ensured that it still killed the process. I'll try to test again, if you have any specific test repro so that I can test your case directly too, that'd be great

[gcampbell-msft]

I'm looking at my test repro wit that setting enabled, and I don't see any issue, the ctest.exe invocation seems to be properly killed. Is there something I'm misunderstanding?

[dgazzoni]

@gcampbell-msft I just extract a kind-of minimal reproducible example from one of the repos:

https://github.com/dgazzoni/ctest-repro

Could you please try with that repository?

For me (with VS Code Version: 1.94.2 (Universal), on macOS Sequoia 15.0.1, and the cmake.ctest.allowParallelJobs setting enabled), if I go to the testing pane and press the Run Tests button (the one that looks like a double Play button) and then press the stop button on that same pane, I still get 7 processes called ex1, which is the name of my binary, hogging up 100% of a core each, which I have to kill manually.

There are 25 tests in that project, and my laptop has 16 cores in total, so apparently it did kill some of the processes, but not all of them.

[gcampbell-msft]

In my repro it works, but of course my repro is much simpler, so thank you for the repro. I'll test with your repro, but I do wonder if it has to do with child processes that have been created not being properly terminated. I'll try your repro as soon as I can, and when I get access to a Mac to test on. Do you know or have the ability to confirm whether this repros on Windows?

[gcampbell-msft]

If you have time and are curious enough, the PR that I've created is here: #4141 and the code where we kill processes is here: https://github.com/microsoft/vscode-cmake-tools/blob/831c33c162b9bac54ebbfde3cfa55f48f98c50d0/src/util.ts#L309.

It does sound like this PR #4141 fixes the spirit of your repro, so I do plan to merge that PR today, as it fixes the attempt to kill, and then we can fix the issue plaguing you separately since it may be a bug / feature gap in our support for killing processes. Thanks

[dgazzoni]

In my repro it works, but of course my repro is much simpler, so thank you for the repro. I'll test with your repro, but I do wonder if it has to do with child processes that have been created not being properly terminated. I'll try your repro as soon as I can, and when I get access to a Mac to test on. Do you know or have the ability to confirm whether this repros on Windows?

I will try to repro on a Windows machine, and I'll get back to you later.

[gcampbell-msft]

@vdilecce @tvatter @MikeWeller PR #4141 has fixed this issue! Thanks for your patience and understanding, and please open a new issue if you hit any other problems.

[Amy-Li03]

Verified this issue on cmake tools v1.20.52(release), It's able to stop running tests through a "Cancel Test Run" button in Testing page.