CodeMaid is deadlocking Visual Studio

[davkean]

Environment

• Visual Studio version: All versions of Visual Studio from 16.11 -> 17.4 Preview
• CodeMaid version: 12.00.300.0
• Code language: Unknown

Description

Our telemetry shows that the CodeMaid extension is hanging Visual Studio, causing folks to use TaskMgr to tear it down. Over the past 21 days, 17.3 has had (sampled) 124 hits, putting it at #90 hang overall in Visual Studio.

Steps to recreate

This is from telemetry data, there are no repro steps.

UI thread is blocked waiting on a lock:

 	[Waiting on lock owned by Thread 26940, double-click or press enter to switch to thread]	
 	ntdll.dll!_ZwWaitForMultipleObjects@20() Line 825	Unknown
>	KERNELBASE.dll!WaitForMultipleObjectsEx(unsigned long nCount, void * const * lpHandles, int bWaitAll, unsigned long dwMilliseconds, int bAlertable) Line 1551	C
 	WindowsBase.ni.dll!511a4f49()	Unknown
 	[Frames below may be incorrect and/or missing, native debugger attempting to walk managed call stack]	
 	WindowsBase.ni.dll!511a4f49()	Unknown
 	[Managed to Native Transition]	
 	WindowsBase.dll!MS.Win32.UnsafeNativeMethods.WaitForMultipleObjectsEx(int nCount, System.IntPtr[] pHandles, bool bWaitAll, int dwMilliseconds, bool bAlertable)	Unknown
 	WindowsBase.dll!System.Windows.Threading.DispatcherSynchronizationContext.Wait(System.IntPtr[] waitHandles, bool waitAll, int millisecondsTimeout)	Unknown
 	mscorlib.dll!System.Threading.SynchronizationContext.InvokeWaitMethodHelper(System.Threading.SynchronizationContext syncContext, System.IntPtr[] waitHandles, bool waitAll, int millisecondsTimeout)	Unknown
 	[Native to Managed Transition]	
 	[Managed to Native Transition]	
 	mscorlib.dll!System.Lazy<EnvDTE.vsCMAccess>.LazyInitValue()	Unknown
 	mscorlib.dll!System.Lazy<EnvDTE.vsCMAccess>.Value.get()	Unknown
 	SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.UI.Converters.CodeItemToImageConverter.GetAccessString(SteveCadwallader.CodeMaid.Model.CodeItems.BaseCodeItemElement codeItem)	Unknown
 	SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.UI.Converters.CodeItemToImageConverter.BuildImageURIString(SteveCadwallader.CodeMaid.Model.CodeItems.BaseCodeItem codeItem)	Unknown
 	SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.UI.Converters.CodeItemToImageConverter.Convert(object value, System.Type targetType, object parameter, System.Globalization.CultureInfo culture)	Unknown

Background thread is holding that lock (within Lazy<T>) and trying to switch to UI thread resulting in deadlock:

 	ntdll.dll!_ZwWaitForMultipleObjects@20() Line 825	Unknown
 	KERNELBASE.dll!WaitForMultipleObjectsEx(unsigned long nCount, void * const * lpHandles, int bWaitAll, unsigned long dwMilliseconds, int bAlertable) Line 1551	C
 	mscorlib.ni.dll!733b42c8()	Unknown
 	[Managed to Native Transition]	
 	mscorlib.dll!System.Threading.Monitor.Wait(object obj, int millisecondsTimeout, bool exitContext)	Unknown
 	mscorlib.dll!System.Threading.Monitor.Wait(object obj, int millisecondsTimeout)	Unknown
 	mscorlib.dll!System.Threading.ManualResetEventSlim.Wait(int millisecondsTimeout, System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.Tasks.Task.SpinThenBlockingWait(int millisecondsTimeout, System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.Tasks.Task.InternalWait(int millisecondsTimeout, System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.Tasks.Task.Wait(int millisecondsTimeout, System.Threading.CancellationToken cancellationToken)	Unknown
 	mscorlib.dll!System.Threading.Tasks.Task.Wait(System.TimeSpan timeout)	Unknown
 	[Waiting on Async Operation, double-click or press enter to view Async Call Stacks]	
 	Microsoft.VisualStudio.Threading.dll!Microsoft.VisualStudio.Threading.JoinableTaskFactory.WaitSynchronouslyCore(System.Threading.Tasks.Task task)	Unknown
 	Microsoft.VisualStudio.Threading.dll!Microsoft.VisualStudio.Threading.JoinableTaskFactory.WaitSynchronously(System.Threading.Tasks.Task task)	Unknown
 	Microsoft.VisualStudio.Threading.dll!Microsoft.VisualStudio.Threading.JoinableTask.CompleteOnCurrentThread()	Unknown
>	SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Helpers.TextDocumentHelper.RunOnUIThread(System.Action action)	Unknown
 	SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Helpers.TextDocumentHelper.GetTextToFirstMatch(EnvDTE.TextPoint startPoint, string matchString)	Unknown
 	SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Helpers.CodeElementHelper.GetMethodDeclaration(EnvDTE.CodeFunction codeFunction)	Unknown
 	SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Helpers.ExplicitInterfaceImplementationHelper.IsExplicitInterfaceImplementation(EnvDTE80.CodeFunction2 codeFunction)	Unknown
 	SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeItems.CodeItemMethod..ctor.AnonymousMethod__6_6()	Unknown
 	SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeItems.BaseCodeItemElement.TryDefault<bool>(System.Func<bool> func)	Unknown
 	SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeItems.BaseCodeItemElement.LazyTryDefault.AnonymousMethod__0()	Unknown
 	mscorlib.dll!System.Lazy<bool>.CreateValue()	Unknown
 	mscorlib.dll!System.Lazy<bool>.LazyInitValue()	Unknown
 	mscorlib.dll!System.Lazy<bool>.Value.get()	Unknown
 	SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeItems.CodeItemMethod..ctor.AnonymousMethod__6_0()	Unknown
 	SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeItems.BaseCodeItemElement.TryDefault<EnvDTE.vsCMAccess>(System.Func<EnvDTE.vsCMAccess> func)	Unknown
 	SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeItems.BaseCodeItemElement.LazyTryDefault.AnonymousMethod__0()	Unknown
 	mscorlib.dll!System.Lazy<EnvDTE.vsCMAccess>.CreateValue()	Unknown
 	mscorlib.dll!System.Lazy<EnvDTE.vsCMAccess>.LazyInitValue()	Unknown
 	mscorlib.dll!System.Lazy<EnvDTE.vsCMAccess>.Value.get()	Unknown
 	SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeItems.BaseCodeItemElement.LoadLazyInitializedValues()	Unknown
 	SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeItems.CodeItemMethod.LoadLazyInitializedValues()	Unknown
 	SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeModelManager.LoadLazyInitializedValues(SteveCadwallader.CodeMaid.Model.CodeModel codeModel)	Unknown
 	SteveCadwallader.CodeMaid.dll!SteveCadwallader.CodeMaid.Model.CodeModelManager.RetrieveAllCodeItemsAsync.AnonymousMethod__0()	Unknown

Current behavior

This code is taking a lock and then attempting to switch to the UI thread while holding that lock. This switch is blocked because the UI thread is currently blocked waiting on that lock.

Expected behavior

• Change LazyThreadSafetyMode.Publication to allow both to run at the same time, which will prevent the UI thread from trying to take lock

-or-

• Switch To AsyncLazy<T> passing in ThreadHelper.JoinableTaskFactory so that the UI thread will allow the background thread to proceed and run on the UI thread if its blocked on it

-or-

• Switch to the UI thread before accessing the Lazy<T>

[codecadwallader]

Thanks for reporting the issue and providing the telemetry that's available. I have a proposed fix in #951 based on the first suggestion. However, I'm not confident that will address the underlying issue. It seems like this would be effective if there were two threads going into the same Lazy initializer but that doesn't appear to be the case from the telemetry. I may mis-understand how the LazyThreadSafetyMode affects other application code though?

[davkean]

Sorry Steve, I missed your reply to this.

It seems like this would be effective if there were two threads going into the same Lazy initializer but that doesn't appear to be the case from the telemetry.

That's the problem, we have two threads trying to get access to the Lazy initializer. When the UI thread wins and the background thread blocks, the product doesn't hang. However, when the background wins and UI thread blocks, then product hangs. This is because UI thread will not "pump" the message that the background thread needs to access the UI thread.

The fix should be good, I haven't seen an update on https://marketplace.visualstudio.com/items?itemName=SteveCadwallader.CodeMaidVS2022 with that version? This continues to rank high and has seen 173 hits past 30 days at a sampling of 10%, so the real numbers are probably close to 1730 hits.