HTML test reports display as ai report with no test execution summary & screenshot when no --output is provided, even with no ai commands

[DavidREntwistle]

Is there an existing issue for this?

• [X] I have searched the existing issues and didn't find mine.

Steps to reproduce

1. Ensure no ai commands are within yourTestFlow.yaml file
2. Run maestro test --format html yourTestFlow.yaml without any --output file location
3. Observe the ~/.maestro/tests/<test-timestamp>/ai-report-yourTestFlow.html file

Actual results

HTML test reports are displaying as ai reports when no --output file location is provided:

AI suggestions for flow yourTestFlow
→ Open other report
[→ Open flow file ( yourTestFlow.yaml )]

0 possible defects found

E.g.:

• maestro test --format html yourTestFlow.yaml = ~/.maestro/tests/<test-timestamp>/ai-report-yourTestFlow.html
• maestro test --format html --output /testFolder/test-report.html yourTestFlow.yaml = ~/testFolder/test-report.html

Expected results

HTML test reports display the test execution summary & screenshot with and without the --output file location. E.g.:

• maestro test --format html yourTestFlow.yaml = ~/.maestro/tests/<test-timestamp>/yourTestFlow.html
• maestro test --format html --output /testFolder/test-report.html yourTestFlow.yaml = ~/testFolder/test-report.html

See image example: https://github.com/mobile-dev-inc/maestro/pull/1750#issuecomment-2128829126

About app

• Native iOS & Android app

About environment

• MacBook Pro M2 Max - 32 GB
• macOS Sonoma 14.7
• arm64

Logs

N/a

Maestro version

1.39.0

How did you install Maestro?

install script (https://get.maestro.mobile.dev)

Anything else?

No response

[linear[bot]]

MA-2346 HTML test reports display as ai report with no test execution summary & screenshot, even with no ai commands

[DavidREntwistle]

Looked into this a bit more, it seems to only work when we include the output e.g. maestro test --format html --output test-report.html yourTestFlow.yaml. This then created the correct flowNameHere.html html report file instead of the ai-report-flowNameHere.html file within the ~/.maestro/tests/ path.

Updated title/details with this info now.

[Fishbowler]

Is the content of the file mostly correct?