[POC] Platform specific input (from device)

[iulianatodoran]

Description

Check if the Click() command can be implemented on the following platforms:

• [x] Windows
• [ ] Android
• [x] MacOS
• [ ] iOS
• [ ] Nintendo

Mocks

Additional info

On Windows we have to consider

• how to put an app in focus
• if there are multiple monitors, how do we handle clicking on coordinates?
• used dll. from https://github.com/michaelnoonan/inputsimulator

On Android What I tried out

• 1. adb shell input tap x y - workable but adb needs to be installed by the user
• 2. searched for dll. that can be imported in Unity - found dll. from Xamarin but it is not compatible with Unity
• 3. Created .aar in Android Studio with Click() on the device and imported it in Unity but it does not have access to other libraries used for input (like uiautomator or others)
• 4. AccessibilityService - followed a tutorial, no conclusions yet

On MacOS

• use AppleScript
• you have to grant access to Script Editor before being able to click
• you can specify app/process that clicks

ProcessStartInfo startInfo = new ProcessStartInfo() { UseShellExecute = true, FileName = "osascript", Arguments = "Click.scpt 1 1" }; Process p = new Process{StartInfo=startInfo}; p.Start();

on run argv tell application "System Events" click at {item 1 of argv, item 2 of argv} end tell end run