EOFError when accessing WindowSpecification for non-existent window via RPyC

[maaboo]

Expected Behavior

WindowSpecification object is created successfully and can be accessed via RPyC

Actual Behavior

Traceback (most recent call last):
  File "/Users/user/Documents/nogit/Pywinauto_Rpyc_Sandbox/./pywinauto_rpyc_sandbox.py", line 33, in <module>
    app_window_exists(rpc)
  File "/Users/user/Documents/nogit/Pywinauto_Rpyc_Sandbox/./pywinauto_rpyc_sandbox.py", line 29, in app_window_exists
    dlg_spec = app.window(title='Save as')
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/.pyenv/versions/pdfm.3.11.3/lib/python3.11/site-packages/rpyc/core/netref.py", line 247, in __call__
    return syncreq(_self, consts.HANDLE_CALL, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/.pyenv/versions/pdfm.3.11.3/lib/python3.11/site-packages/rpyc/core/netref.py", line 69, in syncreq
    return conn.sync_request(handler, proxy, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/.pyenv/versions/pdfm.3.11.3/lib/python3.11/site-packages/rpyc/core/protocol.py", line 725, in sync_request
    return _async_res.value
           ^^^^^^^^^^^^^^^^
  File "/Users/user/.pyenv/versions/pdfm.3.11.3/lib/python3.11/site-packages/rpyc/core/async_.py", line 108, in value
    self.wait()
  File "/Users/user/.pyenv/versions/pdfm.3.11.3/lib/python3.11/site-packages/rpyc/core/async_.py", line 53, in wait
    self._conn.serve(self._ttl)
  File "/Users/user/.pyenv/versions/pdfm.3.11.3/lib/python3.11/site-packages/rpyc/core/protocol.py", line 449, in serve
    self._dispatch(data)
  File "/Users/user/.pyenv/versions/pdfm.3.11.3/lib/python3.11/site-packages/rpyc/core/protocol.py", line 399, in _dispatch
    self._dispatch_request(seq, args)
  File "/Users/user/.pyenv/versions/pdfm.3.11.3/lib/python3.11/site-packages/rpyc/core/protocol.py", line 373, in _dispatch_request
    self._send(consts.MSG_REPLY, seq, self._box(res))
  File "/Users/user/.pyenv/versions/pdfm.3.11.3/lib/python3.11/site-packages/rpyc/core/protocol.py", line 298, in _send
    self._channel.send(data)
  File "/Users/user/.pyenv/versions/pdfm.3.11.3/lib/python3.11/site-packages/rpyc/core/channel.py", line 78, in send
    self.stream.write(header + data + self.FLUSHER)
  File "/Users/user/.pyenv/versions/pdfm.3.11.3/lib/python3.11/site-packages/rpyc/core/stream.py", line 288, in write
    count = self.sock.send(data[:self.MAX_IO_CHUNK])
            ^^^^^^^^^^^^^^
  File "/Users/user/.pyenv/versions/pdfm.3.11.3/lib/python3.11/site-packages/rpyc/core/stream.py", line 96, in __getattr__
    raise EOFError("stream has been closed")
EOFError: stream has been closed

Steps to Reproduce the Problem

1. Create Virtual machine with Rpyc and installed Pywinauto module (I used Parallels Desktop for Mac running on M2 chip)
2. Run the following code:

import rpyc

IP_ADDRESS = '10.211.55.25'
RPYC_PORT = 18813

rpc = rpyc.classic.connect(IP_ADDRESS, port=RPYC_PORT)


def example(rpyc_runner):
    """It runs smoothly (to some extent, see comments)."""

    # allow_magic_lookup=False is mandatory since we get EOFError
    # (possibly another bug but may be related: https://github.com/tomerfiliba-org/rpyc/issues/444)
    app = rpyc_runner.modules.pywinauto.application.Application(backend="uia",
                                                                allow_magic_lookup=False).start('notepad.exe')
    # This line probably works because the dialog itself already exists
    dlg_spec = app.window(title='Untitled - Notepad')


def app_window_exists(rpyc_runner, timeout=30):
    """It causes EOFError.

    The same code (without rpyc_runner.modules prefix) launched on the target system doesn't produce any error.
    """

    app = rpyc_runner.modules.pywinauto.application.Application(backend="uia",
                                                                allow_magic_lookup=False).start('notepad.exe')
    # This line probably doesn't work because the dialog doesn't exist yet
    dlg_spec = app.window(title='Save as')


# example(rpc)
app_window_exists(rpc)

(for your conviniene you can comment and uncomment function calls)

Short Example of Code to Demonstrate the Problem

Specifications

• Pywinauto version: 0.6.8
• Python version and bitness: Python 3.11.3 (main, Jun 5 2023, 12:58:38) [Clang 14.0.0 (clang-1400.0.29.202)]
• Platform and OS: RPyC Server (5.3.1): macOS Ventura 13.3.1, RPyC Client (5.3.0): Windows 11 22H2 22621.1841 ARM64

[junkmd]

I think it is quite difficult to run GUI Automation on a remote server, not just only pywinauto. Especially for mouse operations, we have to make sure that we do not create a situation where the mouse pointer is lost. Most of the time, you have to connect to the display and make sure that it does not sleep.

The following document summarizes the remote execution technique. https://pywinauto.readthedocs.io/en/latest/remote_execution.html

If you have a method that has worked for you, please let us know.

[maaboo]

Well, we actually use (and test) virtual machine which is (almost) totally controlled by us, we can do almost anything with a cursor, display or something else. Not sure if it applicable to other remote situations.

I'm planning to debug what's going on when it loses connection. I already created the environment, just need some time to finish another tasks.

[junkmd]

By the way, have you ever executed the script directly (without RPyC) on that virtual machine?

Before delving into remote execution, I am concerned about whether the arguments you are passing to pywinauto's methods were appropriate in the first place.

[maaboo]

I used bare minimum of arguments and of course I checked the code locally as stated in app_window_exists() docstring:

The same code (without rpyc_runner.modules prefix) launched on the target system doesn't produce any error.

[junkmd]

app_window_exists() docstring:

The same code (without rpyc_runner.modules prefix) launched on the target system doesn't produce any error.

Ah, I see, I overlooked the docstring. I’m sorry.

[junkmd]

I summarized what I've been thinking about this issue:

---

RPyC specifications

I'm not well-versed in the specifics of RPyC, so I'd like to confirm if my understanding is correct. As evident from the traceback, the EOFError itself is occurring within rpyc, and the error message says "stream has been closed." From this, I understand that the difficulty in resolving this issue lies in the inability to redirect the output of what might have occurred on the remote machine to the local machine, making it challenging to identify the exact error that occurred on the remote machine.

COM, uia, and remote-exec

When specifying the 'uia' backend, comtypes is used. I suspect there might be another twist to make COM work on the remote computer.

Regarding the creation of COM objects, an overview is provided here: https://learn.microsoft.com/en-us/windows/win32/learnwin32/creating-an-object-in-com

Furthermore, according to https://learn.microsoft.com/en-us/windows/win32/api/combaseapi/nf-combaseapi-cocreateinstance,

Call CoCreateInstance when you want to create only one object on the local system. To create a single object on a remote system, call the CoCreateInstanceEx function.

And from https://learn.microsoft.com/en-us/windows/win32/api/combaseapi/nf-combaseapi-cocreateinstanceex:

[in] pServerInfo

Information about the computer on which to instantiate the object. See COSERVERINFO.

I am also a maintainer of comtypes. Here's my knowledge related to this:

• The comtypes.CoCreateInstance (which is a Python wrapper for the aforementioned CoCreateInstance) used to instantiate IUIAutomation in pywinauto, does not allow passing the machine name/server information.

• On the other hand, comtypes does have also comtypes.CoCreateInstanceEx (which is a Python wrapper for the aforementioned CoCreateInstanceEx).

  • The comtypes.client.CreateObject internally calls this function and implements a function that feels similar to using VBA's CreateObject.

---

Given the complexity arising from OS, COM implementation, and remote execution, untangling this issue will be challenging, I think.

However, if you resolve this issue using the approach you're attempting, it would be very beneficial for the community.

I am also willing to assist in your efforts to find a solution.