DTaaS icon indicating copy to clipboard operation
DTaaS copied to clipboard

Playwright test report cannot be parsed by Code Cov

Open Omarley7 opened this issue 1 year ago • 1 comments

It seems like there are no coverage stats included in the report to CodeCov. See snippet.

Snippet of the report:

This is an extracted report snippet of the uploaded report to CodeCov. Everything but 1 test, has been included.

It starts from

<<<<<< network
# path=client/playwright-report/results.json
{
  "config": {
    "forbidOnly": false,
    "fullyParallel": false,
    "globalSetup": null,
    "globalTeardown": null,
    "globalTimeout": 600000,
    "grep": {},
    "grepInvert": null,
    "maxFailures": 0,
    "metadata": {},
    "preserveOutput": "always",
    "projects": [...],
    "reporter": [
      [
        "html",
        {
          "outputFile": "playwright-report/index.html"
        }
      ],
      [
        "list",
        null
      ],
      [
        "junit",
        {
          "outputFile": "playwright-report/results.xml"
        }
      ],
      [
        "json",
        {
          "outputFile": "playwright-report/results.json"
        }
      ]
    ],
    "reportSlowTests": {
      "max": 5,
      "threshold": 15000
    },
    "configFile": "/home/runner/work/DTaaS-Bachelor-new-GUI/DTaaS-Bachelor-new-GUI/client/playwright.config.ts",
    "rootDir": "/home/runner/work/DTaaS-Bachelor-new-GUI/DTaaS-Bachelor-new-GUI/client/test/e2e",
    "quiet": false,
    "shard": null,
    "updateSnapshots": "missing",
    "version": "1.32.1",
    "workers": 1,
    "webServer": null
  },
  "suites": [
    {
      "title": "Menu.test.ts",
      "file": "Menu.test.ts",
      "column": 0,
      "line": 0,
      "specs": [],
      "suites": [
        {
          "title": "Menu Links from first page (Layout)",
          "file": "Menu.test.ts",
          "line": 15,
          "column": 6,
          "specs": [
            {
              "title": "Menu Links are visible",
              "ok": true,
              "tags": [],
              "tests": [
                {
                  "timeout": 30000,
                  "annotations": [],
                  "expectedStatus": "passed",
                  "projectId": "chromium",
                  "projectName": "chromium",
                  "results": [
                    {
                      "workerIndex": 0,
                      "status": "passed",
                      "duration": 502,
                      "errors": [],
                      "stdout": [],
                      "stderr": [],
                      "retry": 0,
                      "startTime": "2023-04-21T12:43:49.727Z",
                      "attachments": []
                    }
                  ],
                  "status": "expected"
                }
              ],
              "id": "017baff3819fe8223973-1835883432be1750c98b",
              "file": "Menu.test.ts",
              "line": 21,
              "column": 3
            },
          ]
        }
      ]
    }
  ],
  "errors": []
} 

There are a few possible reasons worth looking into. I think the third one is definitely the way to go:

  • :question: The report format cannot be parsed by CodeCov.
  • :x: The report may need to be generated using function calls, but this seems to be more useful during runtime, and is not likely the correct approach to generate a report file to upload.
  • :point_right: This article mentions how to set up test coverage with Babel and Jest. It does require a lot of work but seems to be the correct approach. The same approach is summarized here.

Even though we are not using Webpack, there might be something left inside react-scrpits. So it might not be necessary to re-wire the application to use a babel transpiler configuration.

Omarley7 avatar Apr 21 '23 14:04 Omarley7