Winium is unable to interact with apps in nightly execution on VM

[jaiswalkpraveen]

I am automating two windows application in nightly execution on Jenkins. It's able to launch the application but unable to find the password text box. It's waiting for it and fail the test cases. I have disabled the VM screen to go into sleep as earlier this was failing due to screen go into sleep mode. Jenkins is configured on Virtual machine. When I am executing this job on Jenkins VM while I am active on the that machine, it's working perfect. Could anyone help me to find out the root cause?

[jgkahua]

This is due to the state of the VM. Before I propose a solution, I need to know the architecture of the nodes you are running the tests on. Are you using your machine or another machine to host the VMs, then you just have Jenkins run the tests on those VMs (nodes)?

In any type of UI automation, the GUI has to be in view, so it seems the configuration of the VMs state is the cause. The test works perfectly when you are logged in because WinAppDriver can see the elements.

I would ensure that in the remote session, you disable locking of the screen on the VMs.

[PraveenITT]

@jgkahua

1. The GUI is in view. I recorded the screen. Winium is able to launch the application but unable to interact with it in nightly jobs. same architecture is working fine when I am on VM using RDC.
2. In nightly job, my machine or any machine not hosting Jenkins VM. If I am hosting it, then it's working perfect. Is it necessary to host the VM for nightly exceution or assign any desktop/machine to it? If I am assigning that VM a physical machine or loging using RDC, it run smoothly.
3. I have disabled locking the VM.

[jgkahua]

So in that case, you need a PHYSICAL machine that can host the VMs (RDP into the machines) and run the tests. You need a physical mouse and keyboard that can run the tests.

Solution Proposed: Have a laptop or desktop RDP into the machine you want to run the tests on, lock your laptop screen, but make sure the RDP session does not timeout (make sure the remote machine doesn't go to sleep). When you lock your screen on your PHYSICAL machine, it should be able to run these tests.

[PraveenITT]

Thanks for response. So yours mean, desktop application automation script need physical VM to execute? As I'm executing web application script and it's running flawless on same machine without assigning a machine to that VM. Is it safe to say, to automate desktop app one need to assign physical machine to VM?

The solution you proposed, I already tried and it's working.

Could you please suggest any setup/architecture where I don't have to assign physical machine to VM for executing nightly jobs?

[jgkahua]

Sadly not, when it comes to UI automation, (any UI automation), usually automation drivers REQUIRE access to a physical keyboard and mouse or touch screen accessibility. Whether it is Windows, Mac OS, Linux, Android, or iOS client, it is crucial to have a physical machine that the drivers can access a keyboard and mouse on.

It doesn't matter whether it is a web application or desktop app or mobile app, they all need the ability to access the screen with some type of peripheral.

You can RDP all day long, but you need a physical machine to host the VMs on that the drivers can use the keyboard and mouse on.

Honestly, are you working with a desktop app on Windows (WPF or UWP or some other type)? IF that is the case, I would recommend migrated over to @WinAppDriver by Microsoft. I converted about a month ago, literally nothing is any different other than NuGet packages, driver instantiation, and some minor objects of type WindowsElement. Winium has minimal support nowadays and documentation.

https://github.com/microsoft/WinAppDriver

[PraveenITT]

Thanks for explanation. It's desktop application and I'm using Winium. I got your point but luckily I'm able to automate web application without hosting Jenkins VM. I'd love to hear more on same how is it possible in case of web application and what's stopping it in case of desktop application.

[psopetto]

I tried the following and it has worked properly for me -> https://support.smartbear.com/testcomplete/docs/testing-with/running/via-rdp/keeping-computer-unlocked.html

This does not require you to have a "PHYSICAL machine that can host the VMs (RDP into the machines)". I do not know if you have solved your issue or not, but I leave it here for anyone else who ends up here looking for an answer.