microsoft
[MTP] Error when VSTest arguments are passed via old `dotnet test` when they will be ignored
We should update the InvokeTestingPlatform target so that it produces an error when VSTest-specific command-line switches are used. We will add a property to opt-out from the error.
These properties should be:
-
VSTestSetting -
VSTestListTests -
VSTestTestCaseFilter -
VSTestTestAdapterPath -
VSTestLogger -
VSTestDiag -
VSTestResultsDirectory -
VSTestCollect -
VSTestBlame -
VSTestBlameCrash -
VSTestBlameHang
- This is a breaking change, but I believe it's for the good. We have seen many users confused in the past because they do
dotnet test --filter somethingand wondering why the filter is ignored. Ordotnet test --logger trx, etc.. - It's not ideal that MTP will be aware of these VSTest-specific properties, but I think it's minor in this case.
This approach allows us to still keep dotnet test (without any argument) to work in mixed mode.
FYI: @OsirisTerje @rprouse @thomhurst @bradwilson
Good idea, better do it now than later after release, because we consume (and have to consume) all parameters because of integrationg with msbuild (do we also have other reasons, like the dynamic parameter providers)?
the test apps themselves can have different parameters based on their extensions, so we cannot just easily validate right at the parameter parsing time (in dotnet test) what is the correct parameter and what is not a correct parameter.
@nohwnd Ah. All those extra arguments goes after the extra --, which ends up in VSTestCLIRunSettings. There is no validation that we do for VSTestCLIRunSettings. These are guaranteed to be sent to the test app, so if anything is invalid, it will explicitly cause the test project to fail. And if the user adds them without -- (e.g, dotnet test --report-trx), then .NET CLI itself will fail with "Unknown switch" error.
The dotnet test process is being improved in .net 10 isn't it?
Will the -- still be required to separate dotnet args from application args?
The dotnet test process is being improved in .net 10 isn't it?
Yes. It will be opt-in via dotnet.config.
Will the -- still be required to separate dotnet args from application args?
If you don't opt-in, then -- is required. If you opt-in, -- must not exist.
We documented both modes in https://learn.microsoft.com/en-us/dotnet/core/testing/unit-testing-with-dotnet-test.
