graphql-client icon indicating copy to clipboard operation
graphql-client copied to clipboard

Published 20 hours ago • verified publisher icon graphql-dotnet

Unstable Integration Tests

Open rose-a opened this issue 5 years ago • 2 comments

The following tests fail randomly on the test job in GitHub Actions:

I've not yet been able to reproduce this on my development machines.

rose-a avatar Feb 12 '20 06:02 rose-a 

Test run for /home/runner/work/graphql-client/graphql-client/tests/GraphQL.Integration.Tests/bin/Release/netcoreapp3.1/GraphQL.Integration.Tests.dll(.NETCoreApp,Version=v3.1)
2020-03-04T14:35:18.8409254Z Microsoft (R) Test Execution Command Line Tool Version 16.3.0
2020-03-04T14:35:18.8412611Z Copyright (c) Microsoft Corporation.  All rights reserved.
2020-03-04T14:35:18.8430935Z 
2020-03-04T14:35:18.9592919Z Starting test execution, please wait...
2020-03-04T14:35:19.0052826Z 
2020-03-04T14:35:19.0053788Z A total of 1 test files matched the specified pattern.
2020-03-04T14:35:26.5724924Z [xUnit.net 00:00:06.22]     GraphQL.Integration.Tests.QueryAndMutationTests.Newtonsoft.PostRequestCanBeCancelled [FAIL]
2020-03-04T14:35:27.1115946Z [xUnit.net 00:00:06.76]     GraphQL.Integration.Tests.QueryAndMutationTests.SystemTextJson.PostRequestCanBeCancelled [FAIL]
2020-03-04T14:35:27.1433668Z   X GraphQL.Integration.Tests.QueryAndMutationTests.Newtonsoft.PostRequestCanBeCancelled [649ms]
2020-03-04T14:35:27.1435163Z   Error Message:
2020-03-04T14:35:27.1435904Z    Execution of the action should be less than 0.500s, but it required more than 0.500s and 320.7µs.
2020-03-04T14:35:27.1436003Z   Stack Trace:
2020-03-04T14:35:27.1436101Z      at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message)
2020-03-04T14:35:27.1436268Z    at FluentAssertions.Execution.TestFrameworkProvider.Throw(String message)
2020-03-04T14:35:27.1436382Z    at FluentAssertions.Execution.DefaultAssertionStrategy.HandleFailure(String message)
2020-03-04T14:35:27.1436480Z    at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
2020-03-04T14:35:27.1436592Z    at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
2020-03-04T14:35:27.1436879Z    at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args)
2020-03-04T14:35:27.1437027Z    at FluentAssertions.Specialized.ExecutionTimeAssertions.BeLessThan(TimeSpan maxDuration, String because, Object[] becauseArgs)
2020-03-04T14:35:27.1437511Z    at GraphQL.Integration.Tests.QueryAndMutationTests.Base.PostRequestCanBeCancelled() in /home/runner/work/graphql-client/graphql-client/tests/GraphQL.Integration.Tests/QueryAndMutationTests/Base.cs:line 177
2020-03-04T14:35:27.1523479Z   X GraphQL.Integration.Tests.QueryAndMutationTests.SystemTextJson.PostRequestCanBeCancelled [510ms]
2020-03-04T14:35:27.1523636Z   Error Message:
2020-03-04T14:35:27.1524234Z    Execution of the action should be less than 0.500s, but it required more than 0.500s and 123.5µs.
2020-03-04T14:35:27.1531251Z   Stack Trace:
2020-03-04T14:35:27.1531432Z      at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message)
2020-03-04T14:35:27.1531559Z    at FluentAssertions.Execution.TestFrameworkProvider.Throw(String message)
2020-03-04T14:35:27.1531666Z    at FluentAssertions.Execution.DefaultAssertionStrategy.HandleFailure(String message)
2020-03-04T14:35:27.1531845Z    at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
2020-03-04T14:35:27.1531962Z    at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
2020-03-04T14:35:27.1532081Z    at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args)
2020-03-04T14:35:27.1532215Z    at FluentAssertions.Specialized.ExecutionTimeAssertions.BeLessThan(TimeSpan maxDuration, String because, Object[] becauseArgs)
2020-03-04T14:35:27.1533180Z    at GraphQL.Integration.Tests.QueryAndMutationTests.Base.PostRequestCanBeCancelled() in /home/runner/work/graphql-client/graphql-client/tests/GraphQL.Integration.Tests/QueryAndMutationTests/Base.cs:line 177
2020-03-04T14:35:28.8645577Z [xUnit.net 00:00:08.52]     GraphQL.Integration.Tests.WebsocketTests.SystemTextJson.CanCreateObservableSubscription [FAIL]
2020-03-04T14:35:29.0264999Z   X GraphQL.Integration.Tests.WebsocketTests.SystemTextJson.CanCreateObservableSubscription [3s 11ms]
2020-03-04T14:35:29.0265988Z   Error Message:
2020-03-04T14:35:29.0266686Z    Expected Subscription to receive new payload, but did not receive an update within 3s
2020-03-04T14:35:29.0267280Z   Stack Trace:
2020-03-04T14:35:29.0267584Z      at FluentAssertions.Execution.XUnit2TestFramework.Throw(String message)
2020-03-04T14:35:29.0267900Z    at FluentAssertions.Execution.TestFrameworkProvider.Throw(String message)
2020-03-04T14:35:29.0268405Z    at FluentAssertions.Execution.DefaultAssertionStrategy.HandleFailure(String message)
2020-03-04T14:35:29.0268818Z    at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
2020-03-04T14:35:29.0270203Z    at FluentAssertions.Execution.AssertionScope.FailWith(Func`1 failReasonFunc)
2020-03-04T14:35:29.0270820Z    at FluentAssertions.Execution.AssertionScope.FailWith(String message, Object[] args)
2020-03-04T14:35:29.0271143Z    at FluentAssertions.Execution.GivenSelector`1.FailWith(String message, Object[] args)
2020-03-04T14:35:29.0272578Z    at GraphQL.Client.Tests.Common.Helpers.ObservableTester`1.SubscriptionAssertions`1.HaveReceivedPayload(TimeSpan timeout, String because, Object[] becauseArgs) in /home/runner/work/graphql-client/graphql-client/tests/GraphQL.Client.Tests.Common/Helpers/ObservableTester.cs:line 86
2020-03-04T14:35:29.0273636Z    at GraphQL.Client.Tests.Common.Helpers.ObservableTester`1.SubscriptionAssertions`1.HaveReceivedPayload(String because, Object[] becauseArgs) in /home/runner/work/graphql-client/graphql-client/tests/GraphQL.Client.Tests.Common/Helpers/ObservableTester.cs:line 90
2020-03-04T14:35:29.0274449Z    at GraphQL.Integration.Tests.WebsocketTests.Base.CanCreateObservableSubscription() in /home/runner/work/graphql-client/graphql-client/tests/GraphQL.Integration.Tests/WebsocketTests/Base.cs:line 130
2020-03-04T14:35:29.0274833Z    at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__139_0(Object state)
2020-03-04T14:35:29.0425748Z Test Run Failed.
2020-03-04T14:35:29.0425815Z 
2020-03-04T14:35:29.0429169Z Total tests: 40
2020-03-04T14:35:29.0429431Z      Passed: 37
2020-03-04T14:35:29.0429671Z      Failed: 3
2020-03-04T14:35:29.0429865Z  Total time: 10.0372 Seconds

rose-a avatar Mar 04 '20 20:03 rose-a

Suggestion of @sungam3r in #357:

I recommend you ALWAYS use ConfigureAwait(false) in library code especially when using XUnit package as a test framework because of Xunit.Sdk.MaxConcurrencySyncContext. In my case maximumConcurrencyLevel was 6 and some tests randomly failed. So please look through entire codebase and add ConfigureAwait(false) to awaitable calls. Perhaps it will help.

rose-a avatar Oct 31 '21 07:10 rose-a