inconsistent scroll simulation

[itaibh]

Description

When using the following code, each time the scroll goes roughly to the same place but not exactly, and there is no way to know exactly how much the element was scrolled:

private void scrollTo(int startX, int startY, int endX, int endY) {
    PointerInput finger = new PointerInput(PointerInput.Kind.TOUCH, "finger");
    Sequence scrollAction = new Sequence(finger, 0);
    scrollAction.addAction(finger.createPointerMove(Duration.ofMillis(0), PointerInput.Origin.viewport(), startX, startY));
    scrollAction.addAction(finger.createPointerDown(PointerInput.MouseButton.LEFT.asArg()));
    scrollAction.addAction(finger.createPointerMove(Duration.ofMillis(500), PointerInput.Origin.viewport(), endX, endY));
    // repeat last action as if to make the finger stay in place for 500 ms
    scrollAction.addAction(finger.createPointerMove(Duration.ofMillis(500), PointerInput.Origin.viewport(), endX, endY));
    scrollAction.addAction(finger.createPointerUp(PointerInput.MouseButton.LEFT.asArg()));
}

scrollTo(540, 1579, 540, 172);

On BrowserStack I see the correct request arrives:

[{"id":"finger","type":"pointer","parameters":{"pointerType":"touch"},"actions":[{"duration":0,"x":540,"y":1579,"type":"pointerMove","origin":"viewport"},{"button":0,"type":"pointerDown"},{"duration":500,"x":540,"y":172,"type":"pointerMove","origin":"viewport"},{"duration":500,"x":540,"y":172,"type":"pointerMove","origin":"viewport"},{"button":0,"type":"pointerUp"}]}]

but still each time I run that code on a fresh instance I get a bit different result.

Environment

• Appium version 1.21.0 (on BrowserStack)
• Android version 9.0, Google Pixel 2, UiAutomator2

Details

Please provide more details, if necessary.

Link To Appium Logs

https://gist.github.com/itaibh/d8028aa2f13ce3d16e15194101e9327e

In the Appium logs you'll see I use the Cancel action, but the result is the same as with lifting the finger.

[valfirst]

@itaibh why do you think it's an issue in Appium Java client?

[itaibh]

I don't, it's an issue with Appium server. If I created this issue in the wrong place please let me know.

[itaibh]

Perhaps not even with Appium server but then I don't know what comes closer to the device.

[mykola-mokhnach]

I would say this behaviour is expected. Basically, what this code is doing, it emulates user touches. And usually these are not 100% precise, because they have some inertia. You could also try mobile: gestures to see if that helps. These are basically shortcuts to actions, just refined by Google and supplied with optimal parameters for each particular gesture type: https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/android/android-mobile-gestures.md

[itaibh]

A follow up question if you may: I need to capture the entire element, so I need to stitch several screenshots together. Is there a way to either eliminate the inertia somehow or to know exactly how much the element is scrolled in a W3C compliant way? Remember I want native scrollable element.

[mykola-mokhnach]

Is there a way to either eliminate the inertia somehow

In theory it must be decreased proportionally to the velocity value (e.g. less velocity -> less inertia).

[itaibh]

I tried higher duration values and I still get inconsistent results... what about the other part of my question? is there a way to know where the element is placed after scroll?

[mykola-mokhnach]

Not sure what exactly you want to achieve. Yes, it is possible to fetch element's location

[itaibh]

As I said before, I want to be able to get a full screenshot of an entire element, including obscured parts that need to be scrolled to. For that I need either an API that returns that screenshot (less preferable in cases of RecyclerView) or to know how much was actually scrolled (this can be achieved by knowing the element's position relative to a given point before and after scroll). Until now I've exhausted the trials with scrolling with low inertia by using higher duration values. Which brings me to a need to compute where the element is so I can crop and stitch correctly, but the commands I tried to use to get the element position always get me the same coordinates, no matter how much I scroll. I'm using RecyclerView in my test, maybe that's why it doesn't work?

[mykola-mokhnach]

but the commands I tried to use to get the element position always get me the same coordinates, no matter how much I scroll.

Try to find the element each time after an action is done. It could be it is cached by the accessibility and thus the same coordinates are returned.

Until now I've exhausted the trials with scrolling with low inertia by using higher duration values

Have you also tried the gestures above, like https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/android/android-mobile-gestures.md#mobile-scrollgesture ?

[itaibh]

Have you also tried the gestures above, like https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/android/android-mobile-gestures.md#mobile-scrollgesture ?

No, not yet.

[itaibh]

It seems this method throws an exception:

driver.execute("mobile: dragGesture", ImmutableMap.of(
                "startX", startX, "startY", startY, "endX", endX, "endY", endY));

org.openqa.selenium.UnsupportedCommandException: mobile: dragGesture

[mykola-mokhnach]

Make sure you are using the most recent server version

[itaibh]

It didn't get to the server, the exception was thrown from the Java upon calling the execute command. The server is BrowserStack. The Java bindings are 8.0.0-beta.