feat: network failure simulation using a stub proxy

[NyCodeGHG]

trafficstars

Proposed Changes

This PR adds a new command simulateNetworkFailure which starts a stub proxy which closes all received connections to simulate a network failure.

Testing

Ran this flow on an Android API 34 emulator.

appId: org.wikipedia
---
- simulateNetworkFailure: true
- openLink: https://google.com/teapot
- assertVisible: "Webpage not available"
- back
- simulateNetworkFailure: false
- openLink: https://google.com/teapot
- assertNotVisible: "Webpage not available"

Issues Fixed

[NyCodeGHG]

I'm not sure how this behaves on macOS. I currently don't have access to a mac to test this.

[johnaziz57]

@NyCodeGHG, I have a macOS and I cant test this for you, but could you please help me know how to build maestro locally to execute at least one test?

[NyCodeGHG]

@johnaziz57 sure, you can run ./gradlew installDist and then execute ./maestro-cli/build/install/maestro/bin/maestro make sure you have a path to a JDK 8 in your JAVA_HOME environment variable

[johnaziz57]

Hey @NyCodeGHG , I ran the gradlew command and it works successfully, I ran another script that I wrote before and it worked; but when I tried to run the same script that you have in the Testing section, I got

Failed to parse file: /Users/john/projects/playground-maestro/samples/android-flow.yaml:3
Unrecognized field "simulateNetworkFailure" (class maestro.orchestra.yaml.YamlFluentCommand), not marked as ignorable

Am I missing something?

[NyCodeGHG]

@johnaziz57 I can't reproduce your problem. I'm not sure what could cause this problem :thinking:

[johnaziz57]

@NyCodeGHG , I assume that ./installDist is not enough to have a bin that contains your changes. Is there any other gradle command that you run?

[NyCodeGHG]

@johnaziz57 I only run ./gradlew installDist and then execute ./maestro-cli/build/install/maestro/bin/maestro test my-test-file.yaml

[johnaziz57]

Hey @NyCodeGHG , I have managed to run it. I think the problem was with JAVA 8 and some certificate installation on my machine that was not going correctly hand in had.

I tested the code and it fails but for a different reason. It fails because - assertVisible: "Webpage not available" The error page doesn't show the exact text

In the end I can say that running this on MacOs with Android Emulator is working perfectly.

Great work and thanks for your patience

[axelniklasson]

Hi there @NyCodeGHG, thanks for taking the time to work on this and to raise this PR! Unfortunately this is not something we will go ahead and merge at this time, since we're not planning on adding network manipulating features to maestro. We did experiment with this some time back and came to the conclusion that for now we want maestro to be a blackbox UI testing framework only and not include advanced device modification tooling such as network failure simulation. Again, really appreciate your work on this and hopefully this can be useful in another way outside of maestro.

[johnaziz57]

@axelniklasson is there any plans to include plugins/extensions for Maestro.

Maybe this way you can let people add the feature they want without affecting the goal of the main project 🙇