maestro icon indicating copy to clipboard operation
maestro copied to clipboard

AssertWithAI returning false instead of true

Open alecalma opened this issue 4 months ago • 1 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues and didn't find mine.

Steps to reproduce

Start Pixel 9 Pro API35 from AndroidStudio and install MySports app from PlayStore Login and select ZSC as your favourite club

appId: com.w4a.mysports tags:

  • android --
  • launchApp
  • assertNotVisible: text: "Login"
  • scrollUntilVisible: element: text: TABELLE direction: DOWN
  • assertWithAI: assertion: the ZSC row has a different background than the other rows

Actual results

Screen Shot 2024-10-29 at 11 21 09 AM Screen Shot 2024-10-29 at 11 22 43 AM

Normally the assert with AI works well or it's clear why the assert is failing, however there are cases like the one above where the assert is false and the explanation is wrong.

I tried to replicate it with chatGPT for web and the assert returns true. Can it be an implementation bug from your side?

Expected results

In the screenshots above, I expect the assert to be true.

About app

Native app for android mobile in ReactNative.

About environment

Pixel 9 Pro API35 from AndroidStudio.

Logs

Logs
<!-- Replace this line with your logs. *DO NOT* remove the backticks! -->

Maestro version

1.39.0

How did you install Maestro?

Homebrew

Anything else?

No response

alecalma avatar Oct 29 '24 10:10 alecalma