java-client icon indicating copy to clipboard operation
java-client copied to clipboard

Inconsistent Input Behavior with PointerInput on iOS 17 using Appium Java Client 8.5.0

Open Deceptio opened this issue 1 year ago • 5 comments

Description

I am currently working with iOS 17 and the Appium Java Client version 8.5.0. My goal is to enter three numbers into a numeric field by simulating key presses using the PointerInput class.

Here's the scenario: I'm attempting to use the keyboard to click the numbers 1, 2, and 3, which correspond to the following coordinates on the screen:

  • Number 1: Button at position (0,561)
  • Number 2: Button at position (125,561)
  • Number 3: Button at position (262,561)

However, I'm experiencing inconsistent input behavior. Instead of obtaining the expected sequence of '123', the output I'm getting is '11122'. This occurs despite having experimented with different durations for the pauses between inputs.

No matter how I adjust the pauses, there seems to be no improvement, and the issue persists where the input is erratic and includes more numbers than intended. It's unclear whether the issue lies with the PointerInput implementation or if there's a specific iOS 17 interaction that I'm not accounting for.

Code

public void fasterPINLogin(String numbersText) {
    PointerInput finger = new PointerInput(PointerInput.Kind.TOUCH, "finger");
    Sequence tap = new Sequence(finger, 0);

    if (isIOS()) {
        clicks(finger, tap,0,561);
        clicks(finger, tap,128,561);
        clicks(finger, tap,262,561);
        iosDriver.perform(List.of(tap));
    }
}

private void clicks(PointerInput finger, Sequence tap, int x, int y) {
    tap.addAction(finger.createPointerMove(ofMillis(10), PointerInput.Origin.viewport(), x, y));
    tap.addAction(finger.createPointerDown(LEFT.asArg()));
    tap.addAction(new Pause(finger, ofMillis(500)));
    tap.addAction(finger.createPointerUp(LEFT.asArg()));
    tap.addAction(new Pause(finger, ofMillis(500)));
}`

Deceptio avatar Nov 06 '23 16:11 Deceptio

try

private void clicks(PointerInput finger, Sequence tap, int x, int y) {
    tap.addAction(finger.createPointerMove(ofMillis(10), PointerInput.Origin.viewport(), x, y));
    tap.addAction(finger.createPointerDown(LEFT.asArg()));
    tap.addAction(new Pause(finger, ofMillis(100)));
    tap.addAction(finger.createPointerUp(LEFT.asArg()));
}

mykola-mokhnach avatar Nov 06 '23 17:11 mykola-mokhnach

Sadly, this is not working either. It is very strange; it doesn't even press button 3! However, when I execute only one tap with button 3, it works. The behavior seems to be completely random.

Deceptio avatar Nov 06 '23 23:11 Deceptio

XCTest itself sets the tap interval to 50ms.

mykola-mokhnach avatar Nov 09 '23 14:11 mykola-mokhnach

I would also try to increase the inter-tap interval to 1s

mykola-mokhnach avatar Nov 09 '23 14:11 mykola-mokhnach

No one has been able to reproduce the behavior. I have tried again. Same behavior. Not even running the same code over and over again gives the same results. I try to control the IOS buttons like this

Deceptio avatar Apr 12 '24 12:04 Deceptio