gitextensions icon indicating copy to clipboard operation
gitextensions copied to clipboard

Use Microsoft Testing Platform (MTP)

Open RussKie opened this issue 9 months ago • 17 comments

The following is from GitHub Copilot:

Benefits of Migrating from VSTest to Microsoft Testing Platform (MTP)

Migrating from VSTest to Microsoft Testing Platform (MTP) offers several benefits:

  1. Lightweight and Portable: Microsoft Testing Platform is designed to be a lightweight and portable alternative to VSTest. It can run tests in command line, CI pipelines, Visual Studio Test Explorer, and Visual Studio Code without the need for additional dependencies such as vstest.console or dotnet test.

  2. Embedded in Test Projects: MTP is embedded directly in your test projects, eliminating the need for separate applications to run tests. This simplifies the process and makes it easier to manage and execute tests.

  3. Open Source: Microsoft Testing Platform is open source, allowing developers to contribute to the project and submit issues. The code is available on GitHub in the microsoft/testfx repository.

  4. Versatility: MTP supports unit testing for both desktop and web applications, making it a versatile testing platform that can be used across different types of projects.

  5. Support in MSTest: MTP is supported by the MSTest runner, which can run tests in various contexts such as CI pipelines, CLI, Visual Studio Test Explorer, and VS Code Text Explorer.

  6. Improved Test Execution: MTP offers improved test execution compared to VSTest, making it a more efficient and effective testing platform.

For more detailed information, you can refer to the following sources:

Merge strategy

I agree that the maintainer squash merge this PR (if the commit message is clear).


:black_nib: I contribute this code under The Developer Certificate of Origin.

RussKie avatar Apr 05 '25 08:04 RussKie

Does not work well in VS 17.13.5.

"Visual Studio Test Explorer supports the new test platform starting with version 17.14"

https://learn.microsoft.com/en-us/dotnet/core/testing/microsoft-testing-platform-vs-vstest#visual-studio

pmiossec avatar Apr 06 '25 11:04 pmiossec

Watch out! It crashes / deletes the GitExtensions.settings file in the profile folder!

mstv avatar Apr 06 '25 11:04 mstv

"Visual Studio Test Explorer supports the new test platform starting with version 17.14"

not released yet :(

mstv avatar Apr 06 '25 11:04 mstv

"Visual Studio Test Explorer supports the new test platform starting with version 17.14"

not released yet :(

Yes, that's just what I checked because I was expecting to be already supported. So maybe this PR is a little premature...

pmiossec avatar Apr 06 '25 11:04 pmiossec

:O I didn't check... Works from cli though :)

RussKie avatar Apr 06 '25 12:04 RussKie

It should be working in 17.13.5. The issue you are hitting seems to be https://github.com/nunit/nunit3-vs-adapter/issues/1241.

The documentation looks misleading to me.

Youssef1313 avatar Apr 07 '25 05:04 Youssef1313

It should be working in 17.13.5. The issue you are hitting seems to be nunit/nunit3-vs-adapter#1241.

The documentation looks misleading to me.

Thank you for the information

RussKie avatar Apr 07 '25 06:04 RussKie

I think it's safe to resurrect this change, VS 17.14 has been out for some time now.

RussKie avatar Jul 20 '25 09:07 RussKie

I think it's safe to resurrect this change, VS 17.14 has been out for some time now.

Can we agree on 6.0.1 and merge master to release/6.0 first?

gerhardol avatar Jul 20 '25 19:07 gerhardol

========== Starting test discovery ==========
[GitCommands.Tests.dll] Unhandled exception. System.IO.FileNotFoundException: Could not load file or assembly 'NUnit3.TestAdapter, Version=5.0.0.0, Culture=neutral, PublicKeyToken=4cb40d35494691ac'. Das System kann die angegebene Datei nicht finden.
File name: 'NUnit3.TestAdapter, Version=5.0.0.0, Culture=neutral, PublicKeyToken=4cb40d35494691ac'
   at SelfRegisteredExtensions.AddSelfRegisteredExtensions(ITestApplicationBuilder builder, String[] args)
   at TestingPlatformEntryPoint.Main(String[] args) in D:\Build\gitextensions3_dev\artifacts\Debug\obj\GitCommands.Tests\net9.0-windows\TestPlatformEntryPoint.cs:line 13
   at TestingPlatformEntryPoint.<Main>(String[] args)

Watch out! It crashes / deletes the GitExtensions.settings file in the profile folder!

Have you already avoided using the user's settings file?

mstv avatar Jul 28 '25 17:07 mstv

Rebase and merge before #12550 ?

gerhardol avatar Oct 19 '25 21:10 gerhardol

Watch out! It crashes / deletes the GitExtensions.settings file in the profile folder!

Have you already avoided using the user's settings file?

@RussKie, important! Or else, has this been fixed by an update of MSVS?

mstv avatar Oct 20 '25 18:10 mstv

Are still seeing the issue? MTP is supported in 17.14+.

RussKie avatar Oct 20 '25 21:10 RussKie

  1. For both solutions, dotnet test invocations in CI scripts need to be updated. It seems like there is only a single invocation here, which is also pointing out to the use of AppVeyor.TestLogger, which probably doesn't yet support MTP. The repo https://github.com/spekt/appveyor.testlogger is archived though. Not sure what the status of that logger is. @codito?

@Youssef1313 appveyor testlogger is not maintained, would recommend moving to Junit logger/MTP reporter extension.

codito avatar Oct 21 '25 03:10 codito

  1. For both solutions, dotnet test invocations in CI scripts need to be updated. It seems like there is only a single invocation here, which is also pointing out to the use of AppVeyor.TestLogger, which probably doesn't yet support MTP. The repo spekt/appveyor.testlogger is archived though. Not sure what the status of that logger is. @codito?

@Youssef1313 appveyor testlogger is not maintained, would recommend moving to Junit logger/MTP reporter extension.

Would this be a matter of changing --logger:Appveyor to --logger:JUnit?

RussKie avatar Oct 25 '25 00:10 RussKie

  1. For both solutions, dotnet test invocations in CI scripts need to be updated. It seems like there is only a single invocation here, which is also pointing out to the use of AppVeyor.TestLogger, which probably doesn't yet support MTP. The repo spekt/appveyor.testlogger is archived though. Not sure what the status of that logger is. @codito?

@Youssef1313 appveyor testlogger is not maintained, would recommend moving to Junit logger/MTP reporter extension.

Would this be a matter of changing --logger:Appveyor to --logger:JUnit?

Please add a reference to JunitXml.TestLogger in test project. MTP reporter can be invoked in CI script like dotnet test -- --report-spekt-junit --report-spekt-junit-filename test-result.xml. The Junit xml result can be uploaded to appveyor using powershell or curl (example).

codito avatar Oct 26 '25 10:10 codito

Are still seeing the issue? MTP is supported in 17.14+.

Yes, still - consistently. Run a single test like BugReportFormTests or a few like AzureDevOpsIntegrationTests. (MSVS 17.14.18)

Important: You will need to close all GE instances before restoring the backup of GitExtensions.settings.

mstv avatar Oct 26 '25 19:10 mstv