Intermittent UI test failure - org.mozilla.samples.browser.SmokeTests - loadWebsitesInMultipleTabsTest

[Archaeopteryx]

Firebase Test Run:

https://console.firebase.google.com/project/moz-android-components-230120/testlab/histories/bh.9f526cd30412cc12/matrices/8843940517278328885/executions/bs.6a17475e05d436bd/testcases/1

Stacktrace:

java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at org.mozilla.samples.browser.SmokeTestsKt.verifyWebsiteContent(SmokeTests.kt:132)
at org.mozilla.samples.browser.SmokeTestsKt.access$verifyWebsiteContent(SmokeTests.kt:1)
at org.mozilla.samples.browser.SmokeTests.loadWebsitesInMultipleTabsTest(SmokeTests.kt:79)
at java.lang.reflect.Method.invoke(Native Method)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at androidx.test.ext.junit.runners.AndroidJUnit4.run(AndroidJUnit4.java:154)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:56)
at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:395)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2145)

Build:

https://github.com/mozilla-mobile/android-components/commit/dd936b6d471388883a6192e900dc13dae2c4e01f

┆Issue is synchronized with this Jira Task

[AaronMT]

Again in https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/DV3vk0AVRCGknomx4Y4p3w/runs/0/artifacts/public/logs/live_backing.log

It looks like on last attempt at loading the asset page it instead is loading in Google? 😵‍💫

https://storage.cloud.google.com/android-components_test_artifacts/2022-07-27_22-37-47.494592_QzrI/matrix_0/blueline-28-en-portrait/test_cases/0000.mp4?authuser=0

My guess is that

    onView(withId(mozilla.components.browser.toolbar.R.id.mozac_browser_toolbar_edit_url_view))
        .perform(replaceText(url), pressImeActionButton())

the IME action button event is happening when the field is wiped and thus the default sample-browser action is going to google

[AndiAJ]

Failed 1x on 9/20 Firebase link

java.lang.AssertionError at org.junit.Assert.fail(Assert.java:86) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertTrue(Assert.java:52) at org.mozilla.samples.browser.SmokeTestsKt.verifyWebsiteContent(SmokeTests.kt:132) at org.mozilla.samples.browser.SmokeTestsKt.access$verifyWebsiteContent(SmokeTests.kt:1) at org.mozilla.samples.browser.SmokeTests.loadWebsitesInMultipleTabsTest(SmokeTests.kt:87)

[csadilek]

Moved to bugzilla: https://bugzilla.mozilla.org/show_bug.cgi?id=1794873

Change performed by the Move to Bugzilla add-on.