How to locate elements in desktop application when classname,id,name are missing - winium driver

[vijayalakshmi-ml]

@skyline-gleb

Hi Gleb,

Am using Winium driver for my automating our desktop application. Am trying to locate elements in the desktop application using Inpsect.exe. But am facing 2 issues

1. Some of the elements doesn't have id or name or classname. In that case How can i identify those elements.
2. Few elements have only id . But when i use the id to locate the element (shown below). It doesn't work. It doesn't recognize the element by the id driver.findElementByName("Expand").click();

As per the inspect tool the automation id corresponding to the element is "Expand". Still it doesn't recognize the element and it throws NoSuchElement exception

Please help me out in this issue

[skyline-gleb]

Hi, @viji-123 try a UISpy (https://github.com/2gis/Winium.Cruciatus/tree/master/tools/UISpy) for get element locators. This tool is more reliable now, than Inspect.exe

[Sourabh25]

Is this issue resolved? I am using UI Spy. I am getting an object which is having an id as number. However the id is changed every time I launch the application. Could someone please suggest how i can identify the element. Its a pane object under window which is having a list of panes

[lucianlorens]

I recommend you using the find_element_by_name() instead or by Xpath. :D

[abhi04]

Hey Gleb, I am automating remote desktop . Through Ui spy i am not able to find the xpath of the OK Button please check the image and let me know asap:

Thanks in advance

[vagi2410]

How to identify the object if partial content of its HelpText is known? No other attributes are unique for that object.

I used Inspect.exe to get the Helptext as UISpy did not work in my machine.

[VSKSunny]

@abhi04 you can make use of sikuli jar and reslove ur concern just capture the image and check whether the image exists or not.

[singhbrijyot91]

For my desktop appliction (.NET), winium takes around 2 minutes to identify a single object. I am new to desktop automation, wanted to know whether its the standard time taken by Winium for object identification ?

[bernd-wechner]

For my desktop appliction (.NET), winium takes around 2 minutes to identify a single object. I am new to desktop automation, wanted to know whether its the standard time taken by Winium for object identification ?

Wow, this blew my mind but I clocked 30 minutes trying to find something that wasn't found and I can't fathom why. I did this in python:

from selenium import webdriver
import time

driver = webdriver.Remote(
    command_executor='http://localhost:9999',
    desired_capabilities={
        "debugConnectToRunningApp": 'false',
        "app": MyEXE
    })
children = driver.find_elements_by_xpath('*')

print(f"find_elements_by_xpath('*')")
for child in children:
    dump(child)
    
print(f"Look for a window:")

start = time.time()
try:
    window = driver.find_element_by_X('Y')
except:
    print(f"Failed to find it ...")
    pass
done = time.time()
dump(window)

dump() is just a function I wrote to display a web elements properties that I can find (which are precious few).

What this does is

1. Starts MyEXE fine - tick
2. Dumps a list of IDs which on closer inspection are all Desktop windows, and only one of them is MyEXE's window - total surprise and unwelcome. My driver is for MyEXE and I want MyEXE's window somehow as a starting point for element searches ...
3. Takes 30, yes 30 minutes to reach done = time.time() from start = time.,time() and returns nothing in window regardless of my chocie of X and Y. I choose X and Y based on properties i see with Microsift's inspect.exe. I can see the name, class_name, the name etc. Never does it find anything. I can choose an element on MyEXEs window and use its ID, name or class_name and still 30 minutes and nothing returned. I have experimented with xpaths with no luck.

Totally bizarre behaviour and stupidly slow.

pywinauto, by comparison works like a charm. Opens my app, offers a top_window() function which returns its window and I can find elements ...