aspnetcore icon indicating copy to clipboard operation
aspnetcore copied to clipboard

Published 20 hours ago • verified publisher icon dotnet

Flaky Various Blazor Template timeouts/failures

Open jkotalik opened this issue 3 years ago • 35 comments

It looks like almost any blazor template test can fail due to a shared fixture being used between them and it timing out.

Builds: https://dev.azure.com/dnceng/internal/_build/results?buildId=1033436&view=ms.vss-test-web.build-test-results-tab&runId=32069600&resultId=125477&paneView=dotnet-dnceng.dnceng-build-release-tasks.helix-test-information-tab

https://dev.azure.com/dnceng/public/_build/results?buildId=1035875&view=ms.vss-test-web.build-test-results-tab&runId=32132094&resultId=125303&paneView=debug

System.TimeoutException : Timeout 30000ms exceeded.
=========================== logs ===========================
<launching> C:\h\w\AC6D092A\w\A51C0945\e\ms-playwright\firefox-1225\firefox\firefox.exe -no-remote -headless -profile C:\h\w\AC6D092A\t\playwright_firefoxdev_profile-0AKzH5 -juggler-pipe -silent
<launched> pid=4752
[err] *** You are running in headless mode.
[out]
[out] Juggler listening to the pipe
[out] console.warn: SearchSettings: "get: No settings file exists, new profile?" (new Error("", "(unknown module)"))
[err] JavaScript error: resource://gre/modules/XULStore.jsm, line 66: Error: Can't find profile directory.
[err]
[err] ###!!! [Child][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
[err]
Note: use DEBUG=pw:api environment variable and rerun to capture Playwright logs.

jkotalik avatar Mar 12 '21 16:03 jkotalik

Thanks for contacting us. We're moving this issue to the Next sprint planning milestone for future evaluation / consideration. We will evaluate the request when we are planning the work for the next milestone. To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

ghost avatar Mar 15 '21 13:03 ghost

Also will track related quarantined blazor template tests

  • [ ] https://github.com/dotnet/aspnetcore/issues/30820
  • [ ] https://github.com/dotnet/aspnetcore/issues/30700
  • [ ] https://github.com/dotnet/aspnetcore/issues/30807

HaoK avatar Mar 23 '21 21:03 HaoK

BlazorWasmHostedPwaTemplate_Works Status: 30 days, 89 runs, 41% pass rate. It looks like it was passing from 4/14-4/26 before hitting the current failure 100% of the time:

System.TimeoutException : Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for selector "ul" to be visible
============================================================
Note: use DEBUG=pw:api environment variable to capture Playwright logs.

Tratcher avatar May 07 '21 20:05 Tratcher

Finally got a chance to look at this, looks like things are passing again after todays latest SDK update by @captainsafia , not sure what this is indicating, but looking at some of these failures, the error is due to a missing method exception from string values in Http header dictionary.

@Tratcher @dougbu @javiercn any ideas what might have been out of sync here? Could this be due to IdentityServer4 being in the call stack? But looks like something was busted for 2 weeks until we got an sdk update, might be good to understand the cause in still

[658.589s] [Templates.Test.BlazorWasmTemplateTest] [Information]       System.MissingMethodException: Method not found: 'Microsoft.Extensions.Primitives.StringValues Microsoft.AspNetCore.Http.IHeaderDictionary.get_Authorization()'.
[658.589s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler.HandleAuthenticateAsync()
[658.589s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) in System.Private.CoreLib.dll:token 0x60047e1+0x28
[658.589s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler.HandleAuthenticateAsync() in Microsoft.AspNetCore.Authentication.JwtBearer.dll:token 0x6000032+0x1c
[658.589s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.AspNetCore.Authentication.AuthenticationHandler`1.AuthenticateAsync() in Microsoft.AspNetCore.Authentication.dll:token 0x6000039+0x9c
[658.589s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.AspNetCore.Authentication.AuthenticationService.AuthenticateAsync(HttpContext context, String scheme) in Microsoft.AspNetCore.Authentication.Core.dll:token 0x6000021+0x1ee
[658.589s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.AspNetCore.Authentication.AuthenticationHandler`1.AuthenticateAsync() in Microsoft.AspNetCore.Authentication.dll:token 0x6000039+0x8f
[658.589s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.AspNetCore.Authentication.AuthenticationService.AuthenticateAsync(HttpContext context, String scheme) in Microsoft.AspNetCore.Authentication.Core.dll:token 0x6000021+0x1ee
[658.589s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context) in Microsoft.AspNetCore.Authentication.dll:token 0x6000044+0x2fa
[658.590s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at IdentityServer4.Hosting.BaseUrlMiddleware.Invoke(HttpContext context) in IdentityServer4.dll:token 0x60001df+0x9a
[658.590s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.AspNetCore.Builder.Extensions.MapWhenMiddleware.Invoke(HttpContext context) in Microsoft.AspNetCore.Http.Abstractions.dll:token 0x6000084+0x10c
[658.590s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.AspNetCore.Builder.Extensions.MapMiddleware.Invoke(HttpContext context) in Microsoft.AspNetCore.Http.Abstractions.dll:token 0x600007b+0x1ec
[658.590s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore.MigrationsEndPointMiddleware.Invoke(HttpContext context) in Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore.dll:token 0x600008e+0x2a4
[658.590s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context) in Microsoft.AspNetCore.Diagnostics.dll:token 0x60000aa+0x82

HaoK avatar May 11 '21 22:05 HaoK

IHeaderDictionary.get_Authorization are new properties added in 6.0. I'll ping you.

Tratcher avatar May 11 '21 22:05 Tratcher

@Tratcher and I chatted a bit, and looks like @dougbu you might have fixed this issue already with https://github.com/dotnet/aspnetcore/commit/07c6a62587fb147cefbd5f84c5a308b5e087cb9c#diff-5ae897b6b974eb8d878b7a7635d5d59d478115e73dba09f3832adcbafb7d137a Where we weren't testing templates against the latest aspnet-shared-fx bits. This should be fixed by your PR over the weekend right?

HaoK avatar May 11 '21 23:05 HaoK

We had a window in which we weren't testing templates using the latest ASP.NET runtime and my PR this weekend fixed that issue. If it explains the remaining flakiness, we're good.

dougbu avatar May 11 '21 23:05 dougbu

Thanks @dougbu , I think that explains the drop to 0%, we still have a ways to go on the rest

Status as of 5/11 for the various blazor template tests:

Seems pretty reliable (failed 2 times out of 1698 - last 14 days - 99.88%) Templates.Test.BlazorWasmTemplateTest.BlazorWasmHostedTemplate_AzureActiveDirectoryTemplate_Works

(Unknown because these were 0% for last 2 weeks) Templates.Test.BlazorWasmTemplateTest.BlazorWasmHostedTemplate_IndividualAuth_Works_WithOutLocalDB Templates.Test.BlazorWasmTemplateTest.BlazorWasmHostedPwaTemplate_Works Templates.Test.BlazorWasmTemplateTest.BlazorWasmHostedTemplate_Works

Broken due to missing method exception from 4/26 (sdk update?), seems healthy-ish after 5/11 sdk update

(75-87% last 30 days) Templates.Test.BlazorWasmTemplateTest.BlazorWasmStandaloneTemplate_Works Templates.Test.BlazorWasmTemplateTest.BlazorWasmStandalonePwaTemplate_Works Templates.Test.BlazorServerTemplateTest.BlazorServerTemplateWorks_IndividualAuth Templates.Test.BlazorServerTemplateTest.BlazorServerTemplateWorks_NoAuth

Mostly 30 second timeout failures, waiting for selector "ul" to be visible sporadic PlaywrightSharp.PlaywrightSharpException : net::ERR_CERT_DATABASE_CHANGED at https://localhost:65432 as well Sporadic total work item fail with no obvious errors (something in dotnet publish/build returned error code)

HaoK avatar May 11 '21 23:05 HaoK

Moving this to preview 7, the retry PR once merged should give us more data to see how the tests behave with 3 retries, hopefully that will let us unquarantine some of these, but likely there will be more fixes/tweaks needed to get to an acceptable pass rate.

HaoK avatar Jun 07 '21 21:06 HaoK

Optimistically marking this as test-fixed since until retries most blazor template tests would fail at least once a day, knock on wood but all the blazor template tests have been green for 2+ days.

image

HaoK avatar Jun 09 '21 20:06 HaoK

I'm not sure I follow this issue. If we are sitting on .NET 6 Preview 5 packages right now and running WASM with Individual Accounts template, would that explain why I'm seeing this same error and call stack?

MissingMethodException: Method not found: 'Microsoft.Extensions.Primitives.StringValues Microsoft.AspNetCore.Http.IHeaderDictionary.get_Authorization()'.
Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler.HandleAuthenticateAsync()
System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start<TStateMachine>(ref TStateMachine stateMachine)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder<TResult>.Start<TStateMachine>(ref TStateMachine stateMachine)
Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler.HandleAuthenticateAsync()
Microsoft.AspNetCore.Authentication.AuthenticationHandler<TOptions>.AuthenticateAsync()
Microsoft.AspNetCore.Authentication.AuthenticationService.AuthenticateAsync(HttpContext context, string scheme)
Microsoft.AspNetCore.Authentication.AuthenticationHandler<TOptions>.AuthenticateAsync()
Microsoft.AspNetCore.Authentication.AuthenticationService.AuthenticateAsync(HttpContext context, string scheme)
Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
IdentityServer4.Hosting.BaseUrlMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Builder.Extensions.MapWhenMiddleware.Invoke(HttpContext context)
Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
Microsoft.AspNetCore.Builder.Extensions.MapMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Diagnostics.EntityFrameworkCore.MigrationsEndPointMiddleware.Invoke(HttpContext context)
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

If so, is there a workaround in the meantime? (Stepping down to version 4, disabling a call in startup, etc.). I'd like to test the app a bit until this issue is addressed (assuming it is the same issue).

taylorchasewhite avatar Jul 10 '21 13:07 taylorchasewhite

You're using new packages with an old runtime/sdk. Make sure everything is aligned to the same preview version.

Tratcher avatar Jul 10 '21 14:07 Tratcher

Nah, that was not the issue. All defaulted Microsoft packages provided by the templates were on 6.0.0-preview.5.21301.17. When I downgraded the following (leaving the rest on 6.0.0-preview.5.21301.17):

  • Microsoft.AspNetCore.ApiAuthorization.IdentityServer
  • Microsoft.AspNetCore.Components.WebAssembly.Server

to 6.0.0-preview.4.21253.17, the site worked. I reproduced this with a fresh copy of a .NET 6 Blazor WASM template.

taylorchasewhite avatar Jul 10 '21 17:07 taylorchasewhite

Please share the output of dotnet --info

Tratcher avatar Jul 10 '21 17:07 Tratcher

Ah I see what you're referring to--runtime mismatch as opposed to the packages being on different versions, my apologies. Here's the output of the dotnet --info command:

C:\Users\white>dotnet --info
.NET SDK (reflecting any global.json):
 Version:   6.0.100-preview.4.21255.9
 Commit:    950e4949a7

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19041
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\6.0.100-preview.4.21255.9\

Host (useful for support):
  Version: 6.0.0-preview.4.21253.7
  Commit:  bfd6048a60

.NET SDKs installed:
  3.1.100 [C:\Program Files\dotnet\sdk]
  3.1.201 [C:\Program Files\dotnet\sdk]
  5.0.300-preview.21258.4 [C:\Program Files\dotnet\sdk]
  5.0.301 [C:\Program Files\dotnet\sdk]
  6.0.100-preview.4.21255.9 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.15 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.16 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.0-preview.4.21253.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.15 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.16 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.0-preview.4.21253.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.15 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.16 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.7 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.0-preview.4.21254.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Is this scenario something that we should always avoid--newer packages without downloading the new runtime? Also--thanks for the info.

taylorchasewhite avatar Jul 10 '21 19:07 taylorchasewhite

Right, always use packages that align to your runtime. Your preview 5 packages depended on new APIs in the preview 5 runtime, but you only have preview 4 installed.

Tratcher avatar Jul 10 '21 20:07 Tratcher

This test also has a duplicate name and is in a class with a duplicate name. See

  • https://github.com/dotnet/aspnetcore/blob/d66bbde41ee3c9d6c8f05746035c0cd50b0f1a93/src/ProjectTemplates/test/BlazorWasmTemplateTest.cs#L84
  • https://github.com/dotnet/aspnetcore/blob/d66bbde41ee3c9d6c8f05746035c0cd50b0f1a93/src/ProjectTemplates/BlazorTemplates.Tests/BlazorWasmTemplateTest.cs#L241

dougbu avatar Jul 20 '21 00:07 dougbu

I think we should close this issue since there's no specific test being tracked here so there's no "unquarantining" that needs to be done. If we see a batch of these test failures, we can reopen this issue. Otherwise, we'll deal with the individual test failures in our routine.

FYI @HaoK @dotnet/aspnet-build

JunTaoLuo avatar Jul 30 '21 01:07 JunTaoLuo

There are still tests pointing to this issue, so the issue shouldn't be closed. Or the tests should be pointing to a different issue. https://github.com/dotnet/aspnetcore/search?q=%22https%3A%2F%2Fgithub.com%2Fdotnet%2Faspnetcore%2Fissues%2F30882%22

BrennanConroy avatar Aug 02 '21 21:08 BrennanConroy

@BrennanConroy I recently opened this one to quarantine some stuff this time around: https://github.com/dotnet/aspnetcore/issues/34941

captainsafia avatar Aug 02 '21 23:08 captainsafia

So the old tests should point to that new issue instead? Basically, we don't want quarantined tests pointing to closed issues.

BrennanConroy avatar Aug 02 '21 23:08 BrennanConroy

FYI Looks like something was changed around 7/9 which caused all of the blazor template tests to fail 100% of the time, this maps roughly to when the templates were updated to start using the implicit uses

HaoK avatar Aug 04 '21 05:08 HaoK

Is this the release build vs debug run issue discussed on teams? https://teams.microsoft.com/l/message/19:[email protected]/1627934962063?tenantId=72f988bf-86f1-41af-91ab-2d7cd011db47&groupId=fdff90ed-0b3b-4caa-a30a-efb4dd47665f&parentMessageId=1627934962063&teamName=DotNet&channelName=ASP.NET%20Build&createdTime=1627934962063

If so, your change looks a bit suspicious https://github.com/dotnet/aspnetcore/commit/dcae2a696a7b76051f69ad7b0ef4541d607d3b3f#diff-d30ee104a4e6fb1f0a52d22fa702c0cde1ec8c308668f2afbabf47af47eccc82L116

BrennanConroy avatar Aug 04 '21 15:08 BrennanConroy

Yeah that's probably it, cc @pranavkm who lead me astray, I'll try a PR to see if that makes the blazor templates happier again https://github.com/dotnet/aspnetcore/pull/34275#discussion_r668121593

HaoK avatar Aug 04 '21 16:08 HaoK

Yeah that's probably it, cc @pranavkm who lead me astray

Serves you right for trusting me 😄

pranavkm avatar Aug 04 '21 16:08 pranavkm

How about these failures?

[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information] fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]       An unhandled exception has occurred while executing the request.
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]       Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 1: 'no such table: Keys'.
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.Data.Sqlite.SqliteException.ThrowExceptionForRC(Int32 rc, sqlite3 db) in Microsoft.Data.Sqlite.dll:token 0x6000138+0x6d
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.Data.Sqlite.SqliteCommand.PrepareAndEnumerateStatements(Stopwatch timer)+MoveNext() in Microsoft.Data.Sqlite.dll:token 0x6000048+0x10a
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.Data.Sqlite.SqliteCommand.GetStatements(Stopwatch timer)+MoveNext() in Microsoft.Data.Sqlite.dll:token 0x600003e+0x145
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.Data.Sqlite.SqliteDataReader.NextResult() in Microsoft.Data.Sqlite.dll:token 0x60000f6+0x17d
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader(CommandBehavior behavior) in Microsoft.Data.Sqlite.dll:token 0x600003d+0xb7
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReaderAsync(CommandBehavior behavior, CancellationToken cancellationToken) in Microsoft.Data.Sqlite.dll:token 0x6000043+0x7
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.Data.Sqlite.SqliteCommand.ExecuteDbDataReaderAsync(CommandBehavior behavior, CancellationToken cancellationToken) in Microsoft.Data.Sqlite.dll:token 0x6000044+0x11
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken) in Microsoft.EntityFrameworkCore.Relational.dll:token 0x600045f+0x2dd
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken) in Microsoft.EntityFrameworkCore.Relational.dll:token 0x600045f+0x587
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.InitializeReaderAsync(AsyncEnumerator enumerator, CancellationToken cancellationToken) in Microsoft.EntityFrameworkCore.Relational.dll:token 0x6001b12+0x136
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync() in Microsoft.EntityFrameworkCore.Relational.dll:token 0x6001b11+0xdb
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToListAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken) in Microsoft.EntityFrameworkCore.dll:token 0x6000218+0xcc
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToListAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken) in Microsoft.EntityFrameworkCore.dll:token 0x6000218+0x159
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToArrayAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken) in Microsoft.EntityFrameworkCore.dll:token 0x6000219+0x6f
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Duende.IdentityServer.EntityFramework.Stores.SigningKeyStore.LoadKeysAsync() in /_/src/EntityFramework.Storage/Stores/SigningKeyStore.cs:line 54
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Duende.IdentityServer.Services.KeyManagement.KeyManager.GetKeysFromStoreAsync(Boolean cache) in /_/src/IdentityServer/Services/Default/KeyManagement/KeyManager.cs:line 419
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Duende.IdentityServer.Services.KeyManagement.KeyManager.GetAllKeysInternalAsync() in /_/src/IdentityServer/Services/Default/KeyManagement/KeyManager.cs:line 103
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Duende.IdentityServer.Services.KeyManagement.KeyManager.GetAllKeysAsync() in /_/src/IdentityServer/Services/Default/KeyManagement/KeyManager.cs:line 90
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Duende.IdentityServer.Services.KeyManagement.AutomaticKeyManagerKeyStore.GetValidationKeysAsync() in /_/src/IdentityServer/Services/Default/KeyManagement/AutomaticKeyManagerKeyStore.cs:line 105
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Duende.IdentityServer.Services.DefaultKeyMaterialService.GetValidationKeysAsync() in /_/src/IdentityServer/Services/Default/DefaultKeyMaterialService.cs:line 106
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Duende.IdentityServer.ResponseHandling.DiscoveryResponseGenerator.CreateDiscoveryDocumentAsync(String baseUrl, String issuerUri) in /_/src/IdentityServer/ResponseHandling/Default/DiscoveryResponseGenerator.cs:line 105
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Duende.IdentityServer.Endpoints.DiscoveryEndpoint.ProcessAsync(HttpContext context) in /_/src/IdentityServer/Endpoints/DiscoveryEndpoint.cs:line 61
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Duende.IdentityServer.Hosting.IdentityServerMiddleware.Invoke(HttpContext context, IEndpointRouter router, IUserSession session, IEventService events, IIssuerNameService issuerNameService, IBackChannelLogoutService backChannelLogoutService) in /_/src/IdentityServer/Hosting/IdentityServerMiddleware.cs:line 84
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Duende.IdentityServer.Hosting.IdentityServerMiddleware.Invoke(HttpContext context, IEndpointRouter router, IUserSession session, IEventService events, IIssuerNameService issuerNameService, IBackChannelLogoutService backChannelLogoutService) in /_/src/IdentityServer/Hosting/IdentityServerMiddleware.cs:line 99
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Duende.IdentityServer.Hosting.MutualTlsEndpointMiddleware.Invoke(HttpContext context, IAuthenticationSchemeProvider schemes) in /_/src/IdentityServer/Hosting/MutualTlsEndpointMiddleware.cs:line 94
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context) in Microsoft.AspNetCore.Authentication.dll:token 0x6000049+0x3be
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Duende.IdentityServer.Hosting.DynamicProviders.DynamicSchemeAuthenticationMiddleware.Invoke(HttpContext context) in /_/src/IdentityServer/Hosting/DynamicProviders/DynamicSchemes/DynamicSchemeAuthenticationMiddleware.cs:line 47
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Duende.IdentityServer.Hosting.BaseUrlMiddleware.Invoke(HttpContext context) in /_/src/IdentityServer/Hosting/BaseUrlMiddleware.cs:line 31
[275.581s] [Templates.Test.BlazorWasmTemplateTest] [Information]          at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context) in Microsoft.AspNetCore.Diagnostics.dll:token 0x60000aa+0x246

Tratcher avatar Aug 05 '21 17:08 Tratcher

Hrm, I thought I fixed those, but maybe I fixed those in the real templates but not the tests since those were disabled, I'll look into those, some migration or db might need to be updated still

HaoK avatar Aug 05 '21 18:08 HaoK

@Tratcher where are you seeing those failures currently? The blazorwasm tempaltes on the quarantined-pr are green again starting from yesterday

HaoK avatar Aug 05 '21 19:08 HaoK

https://dev.azure.com/dnceng/public/_test/analytics?definitionId=331&contextType=build ** Edit ** Templates.Test.BlazorWasmTemplateTest_BlazorWasmHostedTemplate_IndividualAuth_Works_WithOutLocalDB-Chromium

Tratcher avatar Aug 05 '21 21:08 Tratcher

image These have a 0% pass rate, taking a look at the logs they fail while waiting for a "nav" element. Earlier in the logs there is a 404 when requesting _framework/blazor.webassembly.js which I'm guessing is the real issue.

BrennanConroy avatar Oct 29 '21 21:10 BrennanConroy