[Feature] Docs: Add env variable 'PWTEST_BLOB_REPORT_NAME' for blob reports to official documentation or implement different solution than environment variable for this behaviour

[milos-pujic]

trafficstars

I was lucky enough to learn about the environment variable 'PWTEST_BLOB_REPORT_NAME' which enables us to customize the blob report name. It was a deal breaker for our CI setup not being able to customize the blob report name if we were to transition from the unofficial playwright-merge-html-reports package to the playwright's implementation. Our CI has the following matrix setup

  matrix:
    project: [chromium, firefox, webkit]
    shardIndex: [1, 2]
    shardTotal: [2]

with run command

npx playwright test --project=${{ matrix.project }} --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}

With default blob report behaviour at the end we got report-1 and report-2 zip files from the last one which override them, but having the possibility to configure blob name with env variable PWTEST_BLOB_REPORT_NAME='${{ matrix.project }}' made the transition possible, because now we have correct 6 zip files:

all-blob-reports/report-chromium-1.zip all-blob-reports/report-chromium-2.zip all-blob-reports/report-firefox-1.zip all-blob-reports/report-firefox-2.zip all-blob-reports/report-webkit-1.zip all-blob-reports/report-webkit-2.zip

making it possible to have a properly merged HTML report at the end.

After that, I was puzzled as to why this was not part of official documentation and I had to "dig" it out from someone else comment who had picked it out from the code.

Was it added as a workaround for your own pipelines? Is it temporary? Will there be some more appropriate solution like configuration property or similar?

Past references: #26481 and #26508

[yury-s]

Was it added as a workaround for your own pipelines? Is it temporary? Will there be some more appropriate solution like configuration property or similar?

It was added for a little different scenario and we didn't want to publish it in the present form as it ergonomics is suboptimal. For your scenario it sounds like having a reportName option would be sufficient it then could be set to something like that in your code:

"reportName": `report-${process.env.PROJECT}.zip`

[milos-pujic]

It was added for a little different scenario and we didn't want to publish it in the present form as it ergonomics is suboptimal. For your scenario it sounds like having a reportName option would be sufficient it then could be set to something like that in your code:

"reportName": `report-${process.env.PROJECT}.zip`

Thx for the answer. Yes, the proposed solution to have described reportName option would be perfect, but also, with keeping that shard number at the end of the file name. Maybe to omit the extension .zip at the end just to avoid confusion regarding those shard numbers.

[pavelfeldman]

Folding into https://github.com/microsoft/playwright/issues/27284 that has action items.