msbuild
msbuild copied to clipboard
[Bug]: CustomAnalyzerTest tests fail on macOS 13 and 14
Issue Description
Recently PR #10076 added a new CustomAnalyzerTest [Theory] with two sets of data. The PR build checks passed running on a macos-12
image.
However, when I build locally and run tests on macOS 14 on ARM and macOS 13 on Intel, the tests fail.
Steps to Reproduce
- Run
./build.sh
. - Run
./build.sh --test
Expected Behavior
All tests should pass.
Actual Behavior
CustomAnalyzerTest
tests fail.
Failed tests
Microsoft.Build.BuildCheck.UnitTests.EndToEndTests.CustomAnalyzerTest(customAnalyzerNames: [\"CustomAnalyzer\", \"CustomAnalyzer2\"], analysisCandidate: \"AnalysisCandidateWithMultipleAnalyzersInjected\", expectedRegisteredRules: [\"CustomRule1\", \"CustomRule2\", \"CustomRule3\"])
Shouldly.ShouldAssertException : success\r\n should be\r\nTrue\r\n but was\r\nFalse
at Microsoft.Build.BuildCheck.UnitTests.EndToEndTests.BuildAnalyzerRules(TestEnvironment env, String[] customAnalyzerNames) in /Users/jrdodds/Projects/dotnet/msbuild/src/BuildCheck.UnitTests/EndToEndTests.cs:line 174
at Microsoft.Build.BuildCheck.UnitTests.EndToEndTests.CustomAnalyzerTest(String[] customAnalyzerNames, String analysisCandidate, String[] expectedRegisteredRules) in /Users/jrdodds/Projects/dotnet/msbuild/src/BuildCheck.UnitTests/EndToEndTests.cs:line 145
at InvokeStub_EndToEndTests.CustomAnalyzerTest(Object, Span`1)
at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
Microsoft.Build.BuildCheck.UnitTests.EndToEndTests.CustomAnalyzerTest(customAnalyzerNames: [\"CustomAnalyzer\"], analysisCandidate: \"AnalysisCandidate\", expectedRegisteredRules: [\"CustomRule1\", \"CustomRule2\"])
Shouldly.ShouldAssertException : success\r\n should be\r\nTrue\r\n but was\r\nFalse
at Microsoft.Build.BuildCheck.UnitTests.EndToEndTests.BuildAnalyzerRules(TestEnvironment env, String[] customAnalyzerNames) in /Users/jrdodds/Projects/dotnet/msbuild/src/BuildCheck.UnitTests/EndToEndTests.cs:line 174
at Microsoft.Build.BuildCheck.UnitTests.EndToEndTests.CustomAnalyzerTest(String[] customAnalyzerNames, String analysisCandidate, String[] expectedRegisteredRules) in /Users/jrdodds/Projects/dotnet/msbuild/src/BuildCheck.UnitTests/EndToEndTests.cs:line 145
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
The Microsoft.Build.BuildCheck.UnitTests_net8.0_x64.html and .Microsoft.Build.BuildCheck.UnitTests_net8.0_x64xml log files are attached in a .zip.
Microsoft.Build.BuildCheck.UnitTests_net8.0_x64.zip
Analysis
The tests fail in BuildAnalyzerRules()
after calling RunnerUtilities.ExecBootstrapedMSBuild()
:
https://github.com/dotnet/msbuild/blob/393c2fea652873416c8a2028810932a4fa94403f/src/BuildCheck.UnitTests/EndToEndTests.cs#L164-L174
Versions & Configurations
MSBuild version 17.11.0-dev-24263-01+a8e224f80 for .NET 17.11.0.26301
macOS Sonoma 14.4.1 Apple M1 Pro (ARM)
macOS Ventura 13.6.6 Intel Core i7