pact-net icon indicating copy to clipboard operation
pact-net copied to clipboard

Deadlock issue while running test

Open HuiSou opened this issue 1 year ago • 3 comments
trafficstars

Previous issues Have you searched the issue tracker to ensure this hasn't been discussed before?

  • Yes https://github.com/pact-foundation/pact-net/issues?q=is%3Aissue+hangs https://github.com/pact-foundation/pact-net/issues?q=is%3Aissue+freeze https://github.com/pact-foundation/pact-net/issues?q=is%3Aissue+deadlock Version information:
  • OS: Ubuntu 22
  • PactNet Version: 4.5.0
  • .Net Version: 8.0.108
  • Pact Broker Version (if applicable): N/A

Describe the bug I create a pact builder with multiple interaction and try to invoke the call via WebApplicationFactory in my API integration test, and some of my test run are getting stucked and here's the problem I discovered during --blame image *notice the duplicate call of PactNet.Interop.NativeInterop.CleanupMockServer(Int32), this might cause pact_ffi mutex lock each other

Steps To Reproduce

  • Create PACT builder with 15 interaction
  • Create webServer via WebApplicationFactory and change the congfiguration to mockServercontext
  • run api test to get information until it failed, for instance
 while true; do dotnet test  MyApp.Integrationtest --blame-hang  --blame-hang-timeout 10000 ; sleep 1; done
  • once the test hangs, use dotnet-dump to analyze issue

Expected behavior A clear and concise description of what you expected to happen.

  • test should not hangs

Log Output Applicable log output. Please ensure you remove any sensitive information.

00007F04EACEB710 00007f468de3488d [InlinedCallFrame: 00007f04eaceb710] PactNet.Interop.NativeInterop.CleanupMockServer(Int32)
00007F04EACEB710 00007f46199bc919 [InlinedCallFrame: 00007f04eaceb710] PactNet.Interop.NativeInterop.CleanupMockServer(Int32)
00007F04EACEB700 00007F46199BC919 ILStubClass.IL_STUB_PInvoke(Int32)
00007F04EACEB790 00007F46199BC88A PactNet.Drivers.MockServerDriver.ReleaseUnmanagedResources()
00007F04EACEB7B0 00007F46199BC848 PactNet.Drivers.MockServerDriver.Dispose()
00007F04EACEB7D0 00007F4613B9D36C PactNet.PactBuilder+<VerifyAsync>d__9.MoveNext()
00007F04EACEB7F0 00007F4613B9D2D5 PactNet.PactBuilder+<VerifyAsync>d__9.MoveNext()
00007F04EACEB850 00007F461997BF01 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib],[PactNet.PactBuilder+<VerifyAsync>d__9, PactNet]].ExecutionContextCallback(System.Object)
00007F04EACEB870 00007F4613878091 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
00007F04EACEB8C0 00007F461997BDB4 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib],[PactNet.PactBuilder+<VerifyAsync>d__9, PactNet]].MoveNext(System.Threading.Thread)
00007F04EACEB910 00007F461997BCAA System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib],[PactNet.PactBuilder+<VerifyAsync>d__9, PactNet]].MoveNext()
00007F04EACEB930 00007F4613BA9B09 Xunit.Sdk.AsyncTestSyncContext+<>c__DisplayClass7_0.<Post>b__0() [/_/src/xunit.execution/Sdk/AsyncTestSyncContext.cs @ 58]
00007F04EACEB970 00007F4612014FF3 Xunit.Sdk.XunitWorkerThread+<>c.<QueueUserWorkItem>b__5_0(System.Object) [/_/src/common/XunitWorkerThread.cs @ 37]
00007F04EACEB9B0 00007F4613878091 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
00007F04EACEBA00 00007F460E817994 System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread)
00007F04EACEBC50 00007f468da26413 [DebuggerU2MCatchHandlerFrame: 00007f04eacebc50] 

Additional context Add any other context about the problem here.

HuiSou avatar Sep 03 '24 11:09 HuiSou

more information on thread image

HuiSou avatar Sep 03 '24 12:09 HuiSou

Please could you share a code reproduction? This seems very specific to the code you've written so I'd need to see that really.

adamrodger avatar Sep 03 '24 17:09 adamrodger

@adamrodger big thanks for taking a look!, however the problem is in the internal code at my company, but I can setup some PoC to demostrate how it happened later today

btw, as far as I check my error log, it seems that pact mock server got dead during the VerifyAsync Context and pact_ffi is trying to get the server port to cleanup and it hangs if the server is dead

the deadlock is inside pactffi_cleanup_mock_server, as when I remove the native code call in pact-net, everythings works image

HuiSou avatar Sep 04 '24 04:09 HuiSou

@HuiSou I'd still need to see your code I think in order to work out how you're calling this, if you could share that reproduction

adamrodger avatar Oct 31 '24 16:10 adamrodger

Closed due to inactivity

adamrodger avatar Mar 23 '25 16:03 adamrodger