sdk icon indicating copy to clipboard operation
sdk copied to clipboard

[main] Update dependencies from microsoft/testfx

Open dotnet-maestro[bot] opened this issue 3 weeks ago • 7 comments

This pull request updates the following dependencies

From https://github.com/microsoft/testfx

dotnet-maestro[bot] avatar Nov 27 '25 02:11 dotnet-maestro[bot]

/azp run

DonnaChen888 avatar Nov 27 '25 07:11 DonnaChen888

Azure Pipelines successfully started running 2 pipeline(s).

azure-pipelines[bot] avatar Nov 27 '25 07:11 azure-pipelines[bot]

This PR is showing the same problem described here: https://github.com/dotnet/sdk/pull/52028#issuecomment-3618803130

The error log will show:

.dotnet/sdk/10.0.100/NuGet.targets(196,5): error : (NETCORE_ENGINEERING_TELEMETRY=Restore) '[]' is not a valid version string

DonnaChen888 avatar Dec 08 '25 09:12 DonnaChen888

Let's discuss in this thread, as I don't understand how that is the same issue. https://github.com/dotnet/sdk/pull/51895

nagilson avatar Dec 09 '25 19:12 nagilson

@nagilson @MiYanni We should probably revert https://github.com/dotnet/sdk/pull/51934/changes/f7ede1089e8eb02ff28010da968230376f22d408, as the test that was disabled should be fixed with https://github.com/dotnet/sdk/pull/52134, and we don't want to accidentally leave the test disabled

dsplaisted avatar Dec 12 '25 18:12 dsplaisted

Great point, thanks

nagilson avatar Dec 12 '25 19:12 nagilson

If the versions [] issue is fixed, then we expect a new test fx issue to resolve (regex issue) as in the other testfx branches.

nagilson avatar Dec 12 '25 19:12 nagilson

I think we may need to update some of the tests, but still investigating...

》CSC : warning MSTEST0001: Explicitly enable or disable tests parallelization (https://learn.microsoft.com/dotnet/core/testing/mstest-analyzers/mstest0001) [C:\h\w\A9410983\t\dotnetSdkTests\2cwojbxj.idv\6fcc3159-12e0---FBCD72DF\TestProject\TestProject.csproj]
 》C:\h\w\A9410983\t\dotnetSdkTests\2cwojbxj.idv\6fcc3159-12e0---FBCD72DF\TestProject\Test1.cs(17,4): warning MSTEST0025: Use 'Assert.Fail' instead of an always-failing 'Assert.AreEqual' assert (https://learn.microsoft.com/dotnet/core/testing/mstest-analyzers/mstest0025) [C:\h\w\A9410983\t\dotnetSdkTests\2cwojbxj.idv\6fcc3159-12e0---FBCD72DF\TestProject\TestProject.csproj]
 》C:\h\w\A9410983\t\dotnetSdkTests\2cwojbxj.idv\6fcc3159-12e0---FBCD72DF\TestProject\Test1.cs(11,4): warning MSTEST0032: Review or remove the assertion as its condition is known to be always true (https://learn.microsoft.com/dotnet/core/testing/mstest-analyzers/mstest0032) [C:\h\w\A9410983\t\dotnetSdkTests\2cwojbxj.idv\6fcc3159-12e0---FBCD72DF\TestProject\TestProject.csproj]

joeloff avatar Dec 15 '25 19:12 joeloff

~~So looks like the test locally is failing with "exit code 5", but the regex is looking for exit code 8.~~

The test outputs are different. TestProject/OtherTestProject have the expected failures, but AnotherTestProject is not producing the expected results.

joeloff avatar Dec 15 '25 21:12 joeloff

@dotnet/dotnet-testing-admin Could you please take a look at my test fix? Previously, the test relied on inducing 3 different error codes, but by injecting incorrect arguments, all three test projects fail and exit with code 5. By fixing the arguments, the first two tests still fail with different exit codes (2 and 8) and that still passes the validation check to ensure that the dotnet test command exits with code 1 (generic failure).

Below is what the testcase output log shows

Executing 'dotnet test --minimum-expected-tests 2 --configuration Release':
》Running tests from C:\git\sdk\artifacts\tmp\Debug\testing\5f845284-878b---A12CDD09\AnotherTestProject\bin\Release\net10.0\AnotherTestProject.dll (net10.0|x64)
》Running tests from C:\git\sdk\artifacts\tmp\Debug\testing\5f845284-878b---A12CDD09\OtherTestProject\bin\Release\net10.0\OtherTestProject.dll (net10.0|x64)
》Running tests from C:\git\sdk\artifacts\tmp\Debug\testing\5f845284-878b---A12CDD09\TestProject\bin\Release\net10.0\TestProject.dll (net10.0|x64)
》skipped Test2
》  OK skipped!
》  from C:\git\sdk\artifacts\tmp\Debug\testing\5f845284-878b---A12CDD09\OtherTestProject\bin\Release\net10.0\OtherTestProject.dll (net10.0|x64)
》C:\git\sdk\artifacts\tmp\Debug\testing\5f845284-878b---A12CDD09\AnotherTestProject\bin\Release\net10.0\AnotherTestProject.dll (net10.0|x64) passed (113ms)
》failed Test3
》  not OK
》  from C:\git\sdk\artifacts\tmp\Debug\testing\5f845284-878b---A12CDD09\OtherTestProject\bin\Release\net10.0\OtherTestProject.dll (net10.0|x64)
》  not OK
》C:\git\sdk\artifacts\tmp\Debug\testing\5f845284-878b---A12CDD09\OtherTestProject\bin\Release\net10.0\OtherTestProject.dll (net10.0|x64) failed with 1 error(s) (117ms)
》Exit code: 2
》C:\git\sdk\artifacts\tmp\Debug\testing\5f845284-878b---A12CDD09\TestProject\bin\Release\net10.0\TestProject.dll (net10.0|x64) Zero tests ran (108ms)
》Exit code: 8
》
》Test run summary: Failed!
》  C:\git\sdk\artifacts\tmp\Debug\testing\5f845284-878b---A12CDD09\OtherTestProject\bin\Release\net10.0\OtherTestProject.dll (net10.0|x64) failed with 1 error(s) (117ms)
》  C:\git\sdk\artifacts\tmp\Debug\testing\5f845284-878b---A12CDD09\TestProject\bin\Release\net10.0\TestProject.dll (net10.0|x64) Zero tests ran (108ms)
》  C:\git\sdk\artifacts\tmp\Debug\testing\5f845284-878b---A12CDD09\AnotherTestProject\bin\Release\net10.0\AnotherTestProject.dll (net10.0|x64) passed (113ms)
》
》  error: 1
》
》  total: 4
》  failed: 1
》  succeeded: 2
》  skipped: 1
》  duration: 521ms

joeloff avatar Dec 16 '25 18:12 joeloff

@dotnet/dotnet-testing-admin Could you please take a look at my test fix? Previously, the test relied on inducing 3 different error codes, but by injecting incorrect arguments, all three test projects fail and exit with code 5. By fixing the arguments, the first two tests still fail with different exit codes (2 and 8) and that still passes the validation check to ensure that the dotnet test command exits with code 1 (generic failure).

Yes, the update logic is correct. We recently supported --minimum-expected-tests to be on the solution level, so it's very logical that the AnotherTestProject should now pass (because we have more than 2 tests from the 3 projects), I added one assertion for that test project.

mariam-abdulla avatar Dec 18 '25 16:12 mariam-abdulla