Fix issues where the VRT test fails in CI before the tests have all completed

[techanvil]

Bug Description

Unfortunately, the VRT workflow is still failing before all the tests have completed with some regularity.

Examining some recent failures, nearly all of them are failing with following error in one of two permutations:

      COMMAND | Command "test" ended with an error after [450.482s]
      COMMAND | Error: Failed to launch the browser process!

Some of these are followed by the following line, while others are not:

                [1016/142508.119062:ERROR:bus.cc(399)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory

In the sample of test runs there's also one instance where it's failing without the "Failed to launch the browser" error:

      COMMAND | Command "test" ended with an error after [551.962s]
      COMMAND | [object Object]

Example test failures:

• Failed to launch the browser process:
  • https://github.com/google/site-kit-wp/actions/runs/11382083418/job/31664850974#step:5:3397
  • https://github.com/google/site-kit-wp/actions/runs/11372596723/job/31637278108#step:5:5129
  • https://github.com/google/site-kit-wp/actions/runs/11361351132/job/31600869523#step:5:333
  • https://github.com/google/site-kit-wp/actions/runs/11361351132/job/31602935744#step:5:4538
  • https://github.com/google/site-kit-wp/actions/runs/11367605257/job/31620718655#step:5:991
  • https://github.com/google/site-kit-wp/actions/runs/11367605257/job/31621145488#step:5:3227
• Other:
  • https://github.com/google/site-kit-wp/actions/runs/11380954537/job/31661340811#step:5:4080

Steps to reproduce

• Watch for failing VRT test runs in CI.
• They will intermittently fail with the above errors.

Screenshots

Failed to launch browser, with dbus error:

Failed to launch browser, without dbus error:

Other error:

Additional Context

See the previous issue #9406 for the most recently applied fix, and the Additional Context section on https://github.com/google/site-kit-wp/issues/9211 for some relevant avenues that could yet be explored for the fix.

@benbowler please can you update this section if there are avenues you explored from the issue linked above that we don't need to revisit?

---

Do not alter or remove anything below. The following sections will be managed by moderators only.

Acceptance criteria

• VRT test runs should not fail with an error before all the screenshots are captured and the test has run to completion.

Note, additional issues should be raised if needed to follow up on separate avenues of investigation.

Implementation Brief

• [ ] Update the VRT Dockerfile to the following from the #9533 debug PR:
• https://github.com/google/site-kit-wp/blob/c6dc18999f94abda52d5ff2ff4d05d811679f688/docker/backstopjs/Dockerfile#L1-L33
• [ ] Update the tests/backstop/config.js to use the path /usr/bin/chromium for chromium.

Test Coverage

• Approve VRT images as this change of browser version will cause most existing images to fail with minor rendering changes.

QA Brief

• No QA needed after Code Review, but the Code Reviewer should run the test suite several times to ensure it reliably passes.

Changelog entry

• N/A

[benbowler]

Thanks @techanvil, I noticed the reappearance of the browser failed to launch error. It was a little surprising, as in the batch testing the previous fix appeared to prevent this error. I can take a look.

[techanvil]

Thanks @benbowler, yes it's a bit of a shame to see this pop up again as it did seem to have improved for a bit. Thanks for taking a look!

[tofumatt]

(This one was missing an estimate, but since it'll involve running VRT tests, manually approving a lot of changes, and testing the CI runs a few times to verify, I'm marking it as a 7 even though it shouldn't take too long to make the changes.)

IB ✅

[benbowler]

As mentioned in the ticket, no QA required for this ticket, I've run the test suite repeatedly today and it has not failed with the browser launch error.

[tofumatt]

Moving directly to Approval as no QA is needed for this issue.