web icon indicating copy to clipboard operation
web copied to clipboard

Allow for Instanbul coverage with Playwright

Open rokob opened this issue 1 year ago • 3 comments

This makes the PlaywrightLauncherPage function similarly to the ChromeLauncher where we attempt to load the coverage from the browser and return that if availble during the stopSession method. Currently the Playwright launcher breaks early if the native instrumentation is not enabled.

We changed to using the Playwright launcher and were suprised that our coverage stopped working even though we are using the babel istanbul plugin which worked when we were using the ChromeLauncher.

See https://github.com/modernweb-dev/web/blob/master/packages/test-runner-chrome/src/ChromeLauncherPage.ts#L93-L132 for the equivalent code.

rokob avatar Aug 09 '24 21:08 rokob

🦋 Changeset detected

Latest commit: 8484bd3f7ccc3b4b326c3f327484a7ad5e57a230

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@web/test-runner-playwright Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

changeset-bot[bot] avatar Aug 09 '24 21:08 changeset-bot[bot]

Without this change the newly added test fails with:

  71 passing (47s)
  4 pending
  4 failing

  1) test-runner-playwright chromium
       basic
         coverage
           passes coverage test:
     AssertionError: expected undefined to equal 'something'
      at Context.<anonymous> (/Users/andy.weiss/thirdparty/web/integration/test-runner/tests/basic/runBasicTest.ts:36:43)
      at processImmediate (node:internal/timers:476:21)

  2) test-runner-playwright webkit
       basic
         coverage
           passes coverage test:
     AssertionError: expected undefined to equal 'something'
      at Context.<anonymous> (/Users/andy.weiss/thirdparty/web/integration/test-runner/tests/basic/runBasicTest.ts:36:43)
      at processImmediate (node:internal/timers:476:21)

  3) test-runner-playwright firefox
       basic
         coverage
           passes coverage test:
     AssertionError: expected undefined to equal 'something'
      at Context.<anonymous> (/Users/andy.weiss/thirdparty/web/integration/test-runner/tests/basic/runBasicTest.ts:36:43)
      at processImmediate (node:internal/timers:476:21)

  4) test-runner-playwright all
       basic
         coverage
           passes coverage test:
     AssertionError: expected undefined to equal 'something'
      at Context.<anonymous> (/Users/andy.weiss/thirdparty/web/integration/test-runner/tests/basic/runBasicTest.ts:36:43)
      at processImmediate (node:internal/timers:476:21)

and all tests pass locally for me with this change.

rokob avatar Aug 15 '24 01:08 rokob

Awesome. I’ll take a closer look at these tests and assuming CI is also happy we’ll look to get this merged, soon! 🙇‍♂️

Westbrook avatar Aug 15 '24 02:08 Westbrook