Flaky test InjectedStartup_DefaultApplicationNameIsEntryAssembly

[amcasey]

Failing Test(s)

• Microsoft.AspNetCore.Hosting.FunctionalTests.WebHostBuilderTests.InjectedStartup_DefaultApplicationNameIsEntryAssembly

Error Message

Assert.Equal() Failure
                                 ↓ (pos 29)
Expected: ···njectionAssemblyName
Actual:   ···njectionAssemblyName\n[createdump] Gathering state for process···
                                 ↑ (pos 29)

Stacktrace

   at Microsoft.AspNetCore.Hosting.FunctionalTests.WebHostBuilderTests.InjectedStartup_DefaultApplicationNameIsEntryAssembly(TestVariant variant) in /_/src/Hosting/test/FunctionalTests/WebHostBuilderTests.cs:line 64

Logs

[0.001s] [TestLifetime] [Information] Starting test InjectedStartup_DefaultApplicationNameIsEntryAssembly at 2023-11-27T20:42:28
[0.001s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Information] Deploying [Variation] :: ServerType=Kestrel, Runtime=CoreClr, Arch=x64, BaseUrlHint=, Publish=False
[0.001s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Information] Executing dotnet /mnt/vss/_work/1/s/src/Hosting/test/testassets/IStartupInjectionAssemblyName/bin/Release/net9.0/IStartupInjectionAssemblyName.dll --urls http://localhost:40067/ --server Microsoft.AspNetCore.Server.Kestrel
[0.001s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Information] Working directory /mnt/vss/_work/1/s/src/Hosting/test/testassets/IStartupInjectionAssemblyName/bin/Release/net9.0
[0.001s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Information] True
[0.001s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Information] Filename dotnet
[0.001s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Information] False
[0.001s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Information] Arguments /mnt/vss/_work/1/s/src/Hosting/test/testassets/IStartupInjectionAssemblyName/bin/Release/net9.0/IStartupInjectionAssemblyName.dll --urls http://localhost:40067/ --server Microsoft.AspNetCore.Server.Kestrel
[0.002s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Information] Removing environment variable ASPNETCORE_ENVIRONMENT
[0.002s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Information] SET ASPNETCORE_DETAILEDERRORS=true
[0.002s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Information] SET ASPNETCORE_CONTENTROOT=/mnt/vss/_work/1/s/src/Hosting/test/testassets/IStartupInjectionAssemblyName
[0.003s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Information] Started dotnet. Process Id : 13431
[0.003s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Information] Application ready at URL: http://localhost:40067/
[0.149s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Information] dotnet stdout: IStartupInjectionAssemblyName
[0.170s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Warning] dotnet stderr: Unhandled exception. System.InvalidOperationException: Cannot read keys when either application does not have a console or when console input has been redirected. Try Console.Read.
[0.170s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Warning] dotnet stderr:    at System.ConsolePal.ReadKey(Boolean intercept)
[0.171s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Warning] dotnet stderr:    at IStartupInjectionAssemblyName.Program.Main(String[] args) in /mnt/vss/_work/1/s/src/Hosting/test/testassets/IStartupInjectionAssemblyName/Program.cs:line 17
[0.185s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Information] dotnet stdout: [createdump] Gathering state for process 13431 dotnet
[0.186s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Information] dotnet stdout: [createdump] Crashing thread 3477 signal 6 (0006)
[0.205s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Information] Attempting to cancel process 13431
[0.256s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Warning] Unable to terminate the host process with process Id '13431
[0.256s] [Microsoft.AspNetCore.Server.IntegrationTesting.SelfHostDeployer] [Information] [Time]: Total time taken for this test variation '0.2549877' seconds
[0.256s] [TestLifetime] [Information] Finished test InjectedStartup_DefaultApplicationNameIsEntryAssembly in 0.255373s

Build

https://dev.azure.com/dnceng-public/public/_build/results?buildId=481527

[amcasey]

The difference appears to be in log output. If we can't make the assert more robust, we may just want to keep this on the helix retry list (even though it doesn't appear to be a helix test?), but we should at least have an explanation for why it's there.

[BrennanConroy]

Updated with the log.

Looks like a racy test. Should probably remove the Console.ReadLine from the app?

Also, if you want to know how to get logs for non-helix tests I can show you.

[SteveSandersonMS]

This is marked as test-fixed but has failed 30 times in the last 14 days: https://dev.azure.com/dnceng-public/public/_test/analytics?definitionId=86&contextType=build

@MackinnonBuck Should we remove the test-fixed label or do you think something else is going on?

[MackinnonBuck]

@SteveSandersonMS, yeah, let's remove the test-fixed label. I was never able to repro this locally but had a guess about what might be causing the problem and wrote a fix based on that. Looks like my guess was probably wrong!