[QUIC] [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ReadWrite_Success'

[buyaa-n]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=707860 Build error leg or test failing: System.Net.Quic.Functional.Tests.WorkItemExecution Pull request: https://github.com/dotnet/runtime.git/pull/103473 Failing Configuration: net9.0-osx-Debug-x64-coreclr_release-OSX.1200.Amd64.Open

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ReadWrite_Success', Elapsed:",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

From log:

   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ReadWrite_Success', Elapsed: 00:02:10
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ReadWrite_Success', Elapsed: 00:04:10
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ReadWrite_Success', Elapsed: 00:06:11
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ReadWrite_Success', Elapsed: 00:08:12
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ReadWrite_Success', Elapsed: 00:10:13
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ReadWrite_Success', Elapsed: 00:12:14

Known issue validation

Build: :mag_right: https://dev.azure.com/dnceng-public/public/_build/results?buildId=707860 Error message validated: [System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ReadWrite_Success', Elapsed:] Result validation: :x: Known issue did not match with the provided build. Validation performed at: 6/17/2024 11:36:29 AM UTC

Report

Build	Definition	Test	Pull Request
709599	dotnet/runtime	System.Net.Quic.Functional.Tests.WorkItemExecution	dotnet/runtime#103528
707860	dotnet/runtime	System.Net.Quic.Functional.Tests.WorkItemExecution	dotnet/runtime#103473

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	2

[dotnet-policy-service[bot]]

Tagging subscribers to this area: @dotnet/ncl See info in area-owners.md if you want to be subscribed.

[ManickaP]

Happens always on mac: osx.1200.amd64.open, there are some older errors like:

  Discovering: System.Net.Quic.Functional.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Net.Quic.Functional.Tests (found 132 of 147 test cases)
  Starting:    System.Net.Quic.Functional.Tests (parallel test collections = on [6 threads], stop on fail = off)
MsQuic supported and using 'libmsquic.dylib 2.4.0.0 (e0385b047d7bd622319e9f6c4c00184a1294eea8)'.
    System.Net.Quic.Tests.MsQuicPlatformDetectionTests.UnsupportedPlatforms_ThrowsPlatformNotSupportedException [SKIP]
      Condition(s) not met: "IsQuicUnsupported"
    System.Net.Quic.Tests.MsQuicTests.QuicRootedConnectionGetsReleased_ConnectFails [SKIP]
      Condition(s) not met: "IsNotMonoRuntime"
    System.Net.Quic.Tests.MsQuicTests.QuicRootedObjectGetReleased [SKIP]
      Condition(s) not met: "IsNotMonoRuntime"
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ReadWrite_Success', Elapsed: 00:02:01
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ReadWrite_Success', Elapsed: 00:04:02
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ReadWrite_Success', Elapsed: 00:06:03
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ReadWrite_Success', Elapsed: 00:08:03
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ReadWrite_Success', Elapsed: 00:10:04
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ReadWrite_Success', Elapsed: 00:12:05
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ReadWrite_Success', Elapsed: 00:14:06
['System.Net.Quic.Functional.Tests' END OF WORK ITEM LOG: Command timed out, and was killed]

or the 2 newest (14.6. and 16.6.) with the many outputs (that are not failures), containing:

  Discovering: System.Net.Quic.Functional.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Net.Quic.Functional.Tests (found 132 of 147 test cases)
  Starting:    System.Net.Quic.Functional.Tests (parallel test collections = on [6 threads], stop on fail = off)
MsQuic supported and using 'libmsquic.dylib 2.4.0.0 (e0385b047d7bd622319e9f6c4c00184a1294eea8)'.
...
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ConcurrentBidirectionalReadsWrites_Success', Elapsed: 00:02:07
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ConcurrentBidirectionalReadsWrites_Success', Elapsed: 00:04:09
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ConcurrentBidirectionalReadsWrites_Success', Elapsed: 00:06:10
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ConcurrentBidirectionalReadsWrites_Success', Elapsed: 00:08:12
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ConcurrentBidirectionalReadsWrites_Success', Elapsed: 00:10:13
   System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ConcurrentBidirectionalReadsWrites_Success', Elapsed: 00:12:15
['System.Net.Quic.Functional.Tests' END OF WORK ITEM LOG: Command timed out, and was killed]

The tests are using MsQuic from main from 27.2.: https://github.com/microsoft/msquic/commit/e0385b047d7bd622319e9f6c4c00184a1294eea8 We should probably prioritize https://github.com/dotnet/runtime/issues/101590 for 9.0.

[ManickaP]

First confirmed failed 28.5. (still has console log), but might be as old as 13.5. (no console logs anymore, but other attributes of the failure are the same). Query:

Jobs
| join kind=inner WorkItems on JobId
| where Status == "Timeout" and FriendlyName == "System.Net.Quic.Functional.Tests" and not(Source has "/release/")
| project-away JobId, JobName, WorkItemId, JobId1, Name, Build, Creator, System, IsExternal, PullRequestId, Name1
| order by Finished desc 

[rzikm]

I don't see any recent hits in Kusto, so I am inclined to close this issue.

[ManickaP]

I don't see any recent hits in Kusto, so I am inclined to close this issue.

I'm seeing it, e.g.: https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-104543-merge-f453c7dc2ac94248a4/System.Net.Quic.Functional.Tests/1/console.a3c1723f.log?helixlogtype=result

start Mon Jul 8 12:51:50 UTC 2024
...
System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.MsQuicTests.BigWrite_SmallRead_Success', Elapsed: 00:02:09

[ManickaP]

Ok that's a different test, but I found the exact one: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-104539-merge-3fdace35eeea425fa0/System.Net.Quic.Functional.Tests/1/console.3084f80c.log?helixlogtype=result

start Mon Jul 8 08:15:06 EDT 2024
...
System.Net.Quic.Functional.Tests: [Long Running Test] 'System.Net.Quic.Tests.QuicStreamConformanceTests.ReadWrite_Success', Elapsed: 00:12:15

[liveans]

It doesn't repro locally (ran test suite 10hrs straight, no hang), next step is probably creating a PR and collecting more info from CI machine.

[jeffschwMSFT]

removing blocking-clean-ci as this issue is not successfully matching the pattern