[💡 Feature]: Improve mobile automation with WebdriverIO

[christian-bromann]

Is your feature request related to a problem?

I've never done any mobile automation but was recently assigned to update a mobile automation framework. Here are some ideas listed with improvements we can built into WebdriverIO so others don't have to do the same.

Please share your ideas here as well!

Describe the solution you'd like.

Here are some ideas that come to mind:

Native Context

• [x] automatically scroll through the view if element can't be found since in native context element don't show up until they are actually rendered in viewport
• [x] add native dragAndDrop support
• [x] add native longPress support
• [x] add native pinch support
• [x] add native swipe support
• [x] add native tap support
• [x] add native zoom support
• [x] improve web view support
  • [x] improve getContext
  • [x] improve getContexts
  • [x] improve switchContext
• [ ] when using cloud-vendors and providing a local path for appium:app, within the service update the App automatically to their platform and update capabilities
  • [ ] update Sauce Service
  • [ ] update BS-service => @rounak610 , @kamal-kaur04, @07souravkunda , something you can do?
  • [ ] update LambdaTest service => external
• [ ] import common page objects for handling e.g. notifications
• [ ] types for setNetworkConnection are wrong, expects a type when in reality needs a number
• to be continued...

Describe alternatives you've considered.

n/a

Additional context

n/a

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[wdio-bot]

Thanks for reporting!

We greatly appreciate any contributions that help resolve the bug. While we understand that active contributors have their own priorities, we kindly request your assistance if you rely on this bug being fixed. We encourage you to take a look at our contribution guidelines or join our friendly Discord development server, where you can ask any questions you may have. Thank you for your support, and cheers!

[wswebcreation]

Hi @christian-bromann

Thanks for adding this here. A few questions:

automatically scroll through the view if element can't be found since in native context element don't show up until they are actually rendered in viewport

What worked for you? The appium-boilerplate code, or something else? Depending on the app/how the app has been build elements that are not in the view are also not in the source for Android. This is "different" for iOS. iOS should have all elements in the source, but as you can see, it should.

Due to this "behavior" (depending on the app/the tech stack that has been used to build the app) should we create a helper method/extend the scrollIntoView for Native Apps?

when using Sauce Labs and providing a local path for appium:app, within the Sauce Service update the App automatically to their platform and update capabilities

We could help with this, but it is also the "responsibility" of the user / the platform. If we do this for Sauce, we need to do it for all services. Secondly, versioning from the customer's perspective might also be a challenge. They need to be in control and understand which version they want to test. Taking this into consideration I believe that the "logic" would be more complex than the benefits and usage of this feature.

Import common page objects for handling e.g. notifications Can you explain what you mean by this?

[christian-bromann]

Depending on the app/how the app has been build

Do you have more information on how tech stacks differ here?

should we create a helper method/extend the scrollIntoView for Native Apps?

Yeah, extending scrollIntoView makes sense. I was first thinking of extending our middleware but then there are so many cases where you probably don't want to this behavior of "swipe to find an element" to happen.

Taking this into consideration I believe that the "logic" would be more complex than the benefits and usage of this feature.

I agree. At least as a good start, we could have better docs on how to do this.

Can you explain what you mean by this?

I started working on a mobile framework implemented in WebdriverIO , similar to what we have in our Appium boilerplate. I sense that many companies probably attempt to build such framework for themselves. What if we could just provide some well tested page object pattern library. This doesn't need to live in here but e.g. as an extra package, e.g. for handling notifications, navigating through settings etc.

[wswebcreation]

Do you have more information on how tech stacks differ here?

No, not really, at least not docs. We can make it generic, but it might break for others. Flutter is something we might want to skip for now

Yeah, extending scrollIntoView makes sense. I was first thinking of extending our middleware but then there are so many cases where you probably don't want to this behavior of "swipe to find an element" to happen.

Ok, I'll take a look at it, I might need some help in testing this but I'll ping you about that

I agree. At least as a good start, we could have better docs on how to do this.

We then still need to have mobile docs 😓 . Imho users need to be aware of this and check the vendors' docs. Not sure where to put that note/remark in our docs.

What if we could just provide some well tested page object pattern library. This doesn't need to live in here but e.g. as an extra package, e.g. for handling notifications, navigating through settings etc.

My suggestion would be that this is provided by "Appium" through a plugin. Then everyone can leverage this. The downside is that not every vendor allows plugins and people need to be aware that they install plugins. Last but not least, system Notifications and Settings are OS version-dependent. This means a of testing work and maintenance on our side. Happy to pick this up, but then we also need to have good support from cloud vendors to keep testing this.

[christian-bromann]

Ok, I'll take a look at it, I might need some help in testing this but I'll ping you about that

Please note, there is #6536, #6537 and #6538

Not sure where to put that note/remark in our docs.

I am 💯 on having a "Mobile Testing" section with all types of mobile docs we need.

My suggestion would be that this is provided by "Appium" through a plugin.

Interesting thought. How would you expose the interfaces? Would the Appium plugin have specific endpoints to access certain page object elements?

This means a of testing work and maintenance on our side. Happy to pick this up, but then we also need to have good support from cloud vendors to keep testing this.

I am sure we will get test accounts from some of the mobile vendors that support us. I know this may be a lot of maintenance work required. We already do this for the VS Code service where we maintain a large set of page objects to help interact with VS Code elements. I had to give up on the project since VS Code has been changing quite a lot and I couldn't keep up with the maintenance myself. I feel like for iOS and Android the major versions don't change that often, however I want to be mindful about the work we put on ourselves.

[renjieguo927]

Hello,

I really need some documentation related to mobile testing. Currently, on our official website, we need to consider both the web and mobile versions together, which can lead to some APIs being unsuitable for mobile but only applicable to the web. It would be fantastic if we had official mobile documentation.

Additionally, our project is currently using WebdriverIO for automation on both mobile and web. The mobile app we are testing is built with React Native, so it has both iOS and Android versions. On iOS, we usually don’t have to worry about issues with clickable elements due to scroll boxes, as it focuses on the element before clicking. However, on Android, we can’t do that. If the element isn’t visible, we need to write a method to scroll to the element before we can click it.

We have created a utility method to handle this, using UiScrollable (https://developer.android.com/reference/androidx/test/uiautomator/UiScrollable#scrollIntoView(androidx.test.uiautomator.UiObject)) to perform the scrolling. The core of this method is to find the scrollable element and determine whether it is horizontal or vertical. Ultimately, we just need to pass in the test ID to automatically focus on the element and click it. I hope this information is helpful to you.

[wswebcreation]

@renjieguo927

Not sure if you've followed our announcements, but we added a new Mobile section to the docs, see mobile. We've also introduced some new and updated some existing methods. You can now also use 'scrollIntoView' for native iOS and Android.

If you have more methods that you would love to see being added then please let us know here

[christian-bromann]

I will go ahead and close this. Thanks @wswebcreation for leading this effort 🙏 I think creating a page object library would be amazingly valuable but also requires quite some maintenance. If anyone in the community would be interested to take this on, please let us know.