runtime icon indicating copy to clipboard operation
runtime copied to clipboard

System.Net.Security.Tests.SslStreamTlsResumeTests failing

Open kunalspathak opened this issue 1 year ago • 2 comments

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=705166 Build error leg or test failing: System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds Pull request: https://github.com/dotnet/runtime/pull/103159

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": ["ClientCert_DefaultValue_ResumeSucceeds", "Assert.Equal() Failure: Values differ"],
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Mostly due to https://github.com/dotnet/runtime/pull/102656 cc: @rzikm

Known issue validation

Build: :mag_right: https://dev.azure.com/dnceng-public/public/_build/results?buildId=705166 Error message validated: [ClientCert_DefaultValue_ResumeSucceeds Assert.Equal() Failure: Values differ] Result validation: :white_check_mark: Known issue matched with the provided build. Validation performed at: 6/13/2024 7:49:03 AM UTC

Report

Build Definition Test Pull Request
705793 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#101095
705631 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103369
706214 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103382
706355 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103226
705744 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#102834
706320 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#102523
706349 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103399
706344 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103375
706113 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#102736
706298 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#101969
706149 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103143
706263 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103331
705570 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103332
705479 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103241
706227 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103394
706187 dotnet/runtime System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Proxy_Test.MultiProxy_PAC_Failover_Succeeds dotnet/runtime#103364
706174 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientDisableTlsResume_Succeeds dotnet/runtime#103392
706155 dotnet/runtime JIT/opt/ObjectStackAllocation/ObjectStackAllocationTests/ObjectStackAllocationTests.cmd dotnet/runtime#103361
706094 dotnet/runtime JIT/Regression/JitBlue/GitHub_18770/GitHub_18770/GitHub_18770.dll dotnet/runtime#103382
706138 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103391
706088 dotnet/runtime JIT/Regression/JitBlue/GitHub_18770/GitHub_18770/GitHub_18770.dll dotnet/runtime#103382
706127 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103390
706061 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103387
705940 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103206
706039 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103358
706025 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103386
705911 dotnet/runtime WasmTestOnV8-ST-System.Runtime.Tests.WorkItemExecution dotnet/runtime#103381
705977 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103076
705958 dotnet/runtime System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Proxy_Test.MultiProxy_PAC_Failover_Succeeds dotnet/runtime#103364
705952 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#102364
705662 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#99183
705881 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103375
705762 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#100334
705726 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103374
705672 dotnet/runtime JIT/opt/ObjectStackAllocation/ObjectStackAllocationTests/ObjectStackAllocationTests.cmd dotnet/runtime#103361
705719 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103322
705463 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103356
705559 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103279
705536 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103358
705451 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#101876
705416 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103346
705403 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103058
705367 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103352
705344 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#102882
705232 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103195
705321 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103191
705312 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#100356
705283 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103346
705166 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103159
705145 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103341
704882 dotnet/runtime System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds dotnet/runtime#103220

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 0 51

kunalspathak avatar Jun 12 '24 16:06 kunalspathak

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones See info in area-owners.md if you want to be subscribed.

hmm, looks like TLS Resume is not working on TLS 1.3 on Ubuntu 18.04? I will take a look

rzikm avatar Jun 12 '24 16:06 rzikm

@v-wenyuxu is it really the same issue? The test have been disabled, and I don't see this failure in the text of your comment.

rzikm avatar Jul 02 '24 11:07 rzikm

Added some debug print on the related platform and there seems to be something fishy happening in OpenSSL

helixbot@77d5709caac1:/test$ ./dotnet exec --runtimeconfig System.Net.Security.Tests.runtimeconfig.json --depsfile System.Net.Security.Tests.deps.json xunit.console.dll System.Net.Security.Tests.dll -xml testResults.xml -nologo -notrait category=OuterLoop -notrait category=failing -method *NoClientCert_DefaultValue_ResumeSucceeds
  Discovering: System.Net.Security.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Net.Security.Tests (found 1 of 514 test case)
  Starting:    System.Net.Security.Tests (parallel test collections = on [8 threads], stop on fail = off)
Session reused: False
Session reused: False
Adding session to dade675368e24722aa72162292629e68
Adding session to dade675368e24722aa72162292629e68
removing session to dade675368e24722aa72162292629e68
Session reused: False
Session reused: False
setting session
removing session to dade675368e24722aa72162292629e68
Session reused: True
Session reused: True
Adding session to dade675368e24722aa72162292629e68
Session reused: True
Session reused: True
setting session
removing session to @_f�
Session reused: True
Session reused: True
Adding session to @_f�
Session reused: True
Session reused: True
setting session
Session reused: False
Session reused: False
    System.Net.Security.Tests.SslStreamTlsResumeTests.NoClientCert_DefaultValue_ResumeSucceeds(sslProtocol: Tls13) [FAIL]
      Assert.Equal() Failure: Values differ
      Expected: True
      Actual:   False

rzikm avatar Jul 10 '24 13:07 rzikm

The same tests work fine on Ubuntu 20.04 ARM, I believe this is due to a bug in OpenSSL, the version in 18.04 image is stuck on OpenSSL 1.1.1 (no letter in the version) and the image does not receive updates due to being out of support. I think we should keep the problematic combination disabled for this test.

rzikm avatar Jul 10 '24 16:07 rzikm