site-kit-wp icon indicating copy to clipboard operation
site-kit-wp copied to clipboard

Fix failing E2E tests for the WordPress 5.2.21, AMP 1.5.5 test run

Open techanvil opened this issue 1 year ago • 3 comments

Feature Description

There are a number of E2E tests that are failing with some regularity in the WordPress 5.2.21, AMP 1.5.5 test run, and the test run itself is persistently failing.

Taking a few recent failures for runs on develop:

https://github.com/google/site-kit-wp/actions/runs/11345881139/job/31553736684 https://github.com/google/site-kit-wp/actions/runs/11290178675/job/31401485480 https://github.com/google/site-kit-wp/actions/runs/11262285399/job/31317623776 https://github.com/google/site-kit-wp/actions/runs/11334317520/job/31520279033

The specs/modules/analytics/write-scope-requests.test.js suite is failing the most regularly, the following test is failing for all of the above runs:

● Analytics write scope requests › prompts for additional permissions during a new Analytics web data stream creation if the user has not granted the Analytics edit scope

The specs/modules/search-console/dashboard-date-range.test.js suite is also failing with some regularity, the following test fails in two of the above test runs:

● date range filtering on dashboard views › loads new data when the date range is changed on the Site Kit dashboard

The specs/modules/analytics/setup-proxy-no-account-no-tag.test.js suite also fails, just once in the above sample of test runs, with the following test failing:

● setting up the Analytics module with no existing account and no existing tag via proxy › preserves user-filled values provided and auto-submits after approving permissions

These tests should all be fixed so they pass on every test run.


Do not alter or remove anything below. The following sections will be managed by moderators only.

Acceptance criteria

  • The tests detailed in the Feature Description should pass on every test run, both locally and in CI.

Implementation Brief

Note to IB reviewer, PR is already included, after IBR this can go straight into the CR. I also included fix for key metrics jest tests, since it only needs feature flag fix

  • [ ] In specs/modules/analytics/write-scope-requests.test.js
    • Remove this part https://github.com/google/site-kit-wp/blob/1768ae62f46d6a3078345abeeb9c7c8792b2119a/tests/e2e/specs/modules/analytics/write-scope-requests.test.js#L427-L429
    • Add additional page wait after Setup CTA is clicked here https://github.com/google/site-kit-wp/blob/1768ae62f46d6a3078345abeeb9c7c8792b2119a/tests/e2e/specs/modules/analytics/write-scope-requests.test.js#L305-L307
  • [ ] In specs/modules/search-console/dashboard-date-range.test.js
    • Remove .overview-total-impressions .googlesitekit-data-block__datapoint, selector from getTotalImpressions function, as this selector is not present on main or entity dashboard, only in WP dashboard widget, not accessible in this test
    • Await for .googlesitekit-widget selector after visiting the admin page https://github.com/google/site-kit-wp/blob/1768ae62f46d6a3078345abeeb9c7c8792b2119a/tests/e2e/specs/modules/search-console/dashboard-date-range.test.js#L89
  • [ ] Not included in the AC, but since jest tests for key-metrics are failing due to the feature flag, in assets/js/googlesitekit/datastore/user/key-metrics.test.js include conversionReporting feature flag in should return the correct metrics for the %s purpose test case

Test Coverage

  • No Change

QA Brief

  • No QA needed; this should fix E2E tests.

Changelog entry

  • N/A

techanvil avatar Oct 15 '24 15:10 techanvil

From zutigrm - Note, since this is e2e infrastructure related only and marked as high priority , I already included PR, if IB is good, it can go straight to CR. I also covered the jest test fix

binnieshah avatar Oct 18 '24 11:10 binnieshah

IB ✔

eugene-manuilov avatar Oct 21 '24 09:10 eugene-manuilov

Moving directly to Approval as no QA is required.

tofumatt avatar Oct 22 '24 18:10 tofumatt