Make flow execution output work with JUNIT output format

[NyCodeGHG]

Proposed Changes

This PR allows the flow execution output to work while choosing the JUnit output format. It also adds a new flag --hide-execution, which allows hiding of the execution output.

Testing

Ran the android advanced example flow, as well as an internal test suite, using with and without the --output=junit flag.

[axelniklasson]

@NyCodeGHG could you share some more examples on what this change does? Some console output with and without the --hide-execution would be appreciated, thanks!

[NyCodeGHG]

@axelniklasson sure! Previously maestro didn't show the output of a flow when running a single flow using the junit output format, with this change this does work. I added --hide-execution as a feature to get a similar experience to the previous behaviour.

maestro-pr-1721-recording-ffmpeged.webm

[axelniklasson]

@axelniklasson sure! Previously maestro didn't show the output of a flow when running a single flow using the junit output format, with this change this does work. I added --hide-execution as a feature to get a similar experience to the previous behaviour.

maestro-pr-1721-recording-ffmpeged.webm

@NyCodeGHG Nice, this explains the changes very well - thanks! Before we do an in-depth review of this PR, let's invert the behavior of the flag making this an opt-in change instead of an opt-out to keep the current behavior. We can call the flag --show-execution-output and when set enables the output and it defaults to false.

[axelniklasson]

@lucaswiechmann will help get this merged as I'll be OOO starting next week. Please let him know when you've updated the PR @NyCodeGHG!

[NyCodeGHG]

@axelniklasson / @lucaswiechmann are you sure changing this behaviour is a good idea? with this pr it's still different as it would show nothing at all, instead of the output that a flow is running. So the default user experience would be worse

[lucaswiechmann]

@axelniklasson / @lucaswiechmann are you sure changing this behaviour is a good idea? with this pr it's still different as it would show nothing at all, instead of the output that a flow is running. So the default user experience would be worse

Hey @NyCodeGHG

let's keep the current output/behavior that we have today

And then add two flags:

• one to show extended output like this

• Another (we could keep the hide-execution) to not show any output at all

[NyCodeGHG]

@lucaswiechmann I like your suggestion, but I'm not sure if thats easily possible, as the current behaviour/output is an entirely different codepath.

[chrisaydat]

is it possible to add the flow execution results to the output junit xml file?

[bartekpacia]

@chrisaydat Could you clarify what you mean?

[NyCodeGHG]

I'm not interested in this anymore as we moved away from maestro. If someone else wants to pick this up, feel free to create a follow up PR.

[bartekpacia]

I see. Thank you for all the contributions you made to Maestro.

I'll re-open the PR, it's good work and I really want to merge it.

[chrisaydat]

@bartekpacia I mean something like this, whether it is the labels or the results as is on both HTML and Junit reports

[bartekpacia]

Thanks for clarifying @chrisaydat. I think it warrants a new issue, it's not related to this PR.

[bartekpacia]

Review note:

• I very much like the reduced code duplication across TestRunner and TestSuiteInteractor. Related: #1918
• I don't agree with adding a new NoopResultsView

[bartekpacia]

My reasoning: specifying --format (whether JUnit or HTML) should have no impact on Maestro CLI output. It's an unrelated option.

Therefore, I don't think the --hide-execution is needed.

[bartekpacia]

Ahh, there's a big merge conflict in TestCommand, because of #1732