FlaUI icon indicating copy to clipboard operation
FlaUI copied to clipboard

COM exception with active CacheRequest

Open lsim opened this issue 4 years ago • 4 comments

Describe the bug I need to traverse a menu efficiently. I try to use a CacheRequest to speed it up. This causes a COM exception to be thrown:

Error HRESULT E_FAIL has been returned from a call to a COM component.
   at Interop.UIAutomationClient.IUIAutomationElement.FindFirstBuildCache(TreeScope scope, IUIAutomationCondition condition, IUIAutomationCacheRequest cacheRequest)
   at FlaUI.UIA3.UIA3FrameworkAutomationElement.FindFirst(TreeScope treeScope, ConditionBase condition)
   at FlaUI.Core.AutomationElements.AutomationElement.FindFirst(TreeScope treeScope, ConditionBase condition)
   at FlaUI.Core.AutomationElements.AutomationElement.FindFirstChild(ConditionBase condition)
   at FlaUI.Core.AutomationElements.AutomationElement.FindFirstChild(Func`2 conditionFunc)

Code snippets Can reproduce with this class (start notepad first):

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using FlaUI.Core.AutomationElements;
using FlaUI.Core.Conditions;
using FlaUI.Core.Definitions;
using FlaUI.UIA3;

namespace FlauiRepro
{
  internal class Program
  {
    private static readonly UIA3Automation Automation = new UIA3Automation();
    
    public static void Main(string[] args)
    {
      var app = FlaUI.Core.Application.Attach("notepad.exe");
      var window = app.GetMainWindow(Automation);

      DoWithMenuCache(() => ClickFileOpenMenuItem(window, "File", "Open"));

      // Calling without caching works fine:
      // ClickFileOpenMenuItem(window, "File", "Open");
    }
    
    private static void ClickFileOpenMenuItem(Window window, string lvl1, string lvl2)
    {
      var possibleMenus = window.FindAll(TreeScope.Descendants, Automation.ConditionFactory.Menu());
      Log($"Found {possibleMenus.Length} menus");
      if (possibleMenus.Length == 0) return;

      var fileMenuItem = possibleMenus
        .Select(menu => menu.FindFirstChild(cf => cf.ByName(lvl1)))
        .FirstOrDefault(x => x != null);
      // Expand File menu before attempting to traverse
      fileMenuItem?.Patterns.ExpandCollapse.PatternOrDefault?.Expand();
      // We do 'Contains' matching on the name property which seems to require expensive traversal 
      var openMenuItem = fileMenuItem?
        .FindAll(TreeScope.Descendants, TrueCondition.Default)
        .FirstOrDefault(child => child.Name.Contains(lvl2));
      DumpElement(openMenuItem);

      openMenuItem?.Patterns.Invoke.PatternOrDefault?.Invoke();
    }

    private static void DoWithMenuCache(Action fn)
    {
      var cacheRequest = new FlaUI.Core.CacheRequest();
      cacheRequest.Patterns.Add(Automation.PatternLibrary.InvokePattern);
      cacheRequest.Patterns.Add(Automation.PatternLibrary.ExpandCollapsePattern);
      cacheRequest.Properties.Add(Automation.PropertyLibrary.Element.Name);
      cacheRequest.Properties.Add(Automation.PropertyLibrary.Element.ControlType);
      cacheRequest.TreeScope = TreeScope.Element;
      try
      {
        using (cacheRequest.Activate())
        {
          fn.Invoke();
        }
      }
      catch (Exception e)
      {
        Log( "Failed traversing menu with cache\n" + e.Message + Environment.NewLine + e.StackTrace);
      }
    }
    
    private static void DumpElement(AutomationElement e, string indent = "")
    {
      if (e == null)
        Log(indent + "No element!");
      else
        Log(indent + "Name '{0}', Type: '{1}', Patterns: '{2}'", e.Name, e.ControlType, string.Join(", ", e.GetSupportedPatterns().Select(p => p.Name)));
    }

    private static void Log(string format, params object[] args) =>
      Debugger.Log(1, "Debug", string.Format(format + Environment.NewLine, args));
  }
}

Additional context FlaUI.Core 3.2.0 + FlaUI.UIA3 3.2.0

Calling from STA/MTA thread give the same result

lsim avatar Apr 22 '21 06:04 lsim

This issue also happens to me. It seems, that chaining the Find*-methods inside an active Cache request causes the issue.

So instead of

foo.FindFirstChild().FindFirstDescendant("bar")

you need to do this directly

foo.FindFirstDescendant("bar");

which is kind of annoying and probably does not work in all cases.

twity1337 avatar Apr 26 '21 18:04 twity1337

Thanks for the info. Yeah, that workaround doesn't work for menus which often are not created until their parent menu is expanded, so you have to do multiple .FindXxxx() invocations to traverse them.

lsim avatar Apr 29 '21 08:04 lsim

I just played around a bit and found out that the problem didn't occur again after having the automation running on a Worker Thread (before it was running on the Main Thread). So it seems the Thread Apartment State (MTA, instead of STA) might be the reason.

Try to wrap your code into System.Threading.Tasks.Task.Run(() => { /* .... */ }); and try it again. This might solve the problem. If not, I did something wrong while trying to reproduce the error 😅

twity1337 avatar Apr 29 '21 12:04 twity1337

That was the first thing I tried - as indicated (somewhat obscurely) in the initial issue text. The code I pasted, when run as a command line app, will reproduce the issue on an MTA thread - same as with Task.Run.

lsim avatar May 05 '21 10:05 lsim