microsoft
[Feature] support test coverage for component testing
I feel component testing can replace most of the unit tests, so it would be nice if component testing can support test coverage
Playwright is awesome for e2e and is going to be the same for component tests as well. This is one of the essential features for us as we are using the code coverage as one of our quality gates on PRs
have any update? I'm looking forward to getting coverage in a svelte project using playwright
Now our team want to use playwright component testing replace unit tests for ui components, we think code coverage report such as provided by jest is a very important feature.
Related to #7030
it's not working for projects which are not based on babel, there are webpack and CRA (create-react-app) Can we increase priority of the issue?
Just implemented the code coverage report to monocart-reporter Here is an example repository showing component testing for vue: https://github.com/cenfun/playwright-ct-vue
Preview coverage report: https://cenfun.github.io/playwright-ct-vue/coverage/
Using class-coverage API to collect V8 coerage data, so it's chromium only but supported both JS and CSS
Related to https://github.com/microsoft/playwright/issues/22953
Thanks to Coverage Class, we are able to generate coverage for our e2e testing.
@stevez we too felt the same, so we decided to remove unit tests and replace them with component tests. So we need to generate coverage for both unit and component tests, merge, and push to quality gate,
Our application consists of unit tests using Jest and component tests using Playwright, and when we try to merge the code coverage generated by both tools using nyc we are facing the following problem.
We have a file that has 37 functions, one arrow function, one function that has two nested functions, and the remaining are normal functions. When we generate coverage for this file, the count in fnMap is not matching in both json. A function that has two nested functions is considered one single function in component coverage, with unit test it is considered three functions. Since count is not matching, nyc not able to merge. It's just appending.
Thanks to Coverage Class, we are able to generate coverage for our e2e testing.
@stevez we too felt the same, so we decided to remove unit tests and replace them with component tests. So we need to generate coverage for both unit and component tests, merge, and push to quality gate,
Our application consists of unit tests using Jest and component tests using Playwright, and when we try to merge the code coverage generated by both tools using nyc we are facing the following problem.
We have a file that has 37 functions, one arrow function, one function that has two nested functions, and the remaining are normal functions. When we generate coverage for this file, the count in fnMap is not matching in both json. A function that has two nested functions is considered one single function in component coverage, with unit test it is considered three functions. Since count is not matching, nyc not able to merge. It's just appending.
@raju-a, are you using playwright coverage class?(https://playwright.dev/docs/api/class-coverage), I believe this is using v8 coverage, I wonder if you also using v8 in jest instead of istanbul. If jest is using istanbul and playwright is using v8, then the merge is not going to work
I am wondering if PW has a way to instrument the code using Istanbul and not relying on v8 coverage.
@stevez @Tallyb Thanks a lot ! You guys are right! I didn't specify coverageProvider so it took babel as default. After adding coverageProvider:'v8' I was able to merge coverage json partially, I mean f function counts are incremented but still , a function that has two nested functions is considered one single function in component coverage with v8, with unit test coverage(v8) is considered to have three functions. fnMap array count is still not matching, so it merged partially,
In more detail to that file has 37 functions, one arrow function, one function that has two nested functions, and the remaining are normal functions.
in unit test fnMap array size is 37 in case of component tests fnMap array size is 35. In merged coverage json
fnMap array size is 41, before adding coverageProvider:'v8'it was 74.
Do we have a test coverage tool for Playwright-Experimental-React Component test automation?
I tried with @mxschmitt https://github.com/mxschmitt/playwright-test-coverage/tree/main but unfortunately it returns empty folder .nyc_output after execution. baseFixtures.ts covers page, context. But for components, mount is under the hood and unsure how to handle it. Any suggestions please ?
I got code coverage working for react using the example repo from @cenfun https://github.com/cenfun/playwright-ct-vue by just changing any imports from vue to react.
