[Feature]: Test-meta-data of collected tests available in a setup-phase/-project

[tkowalski-exxeta]

🚀 Feature Request

First of all, thanks for all your awesome work on playwright! It's a joy to use!

I am currently working on a project were I need to test using many different users (each with different roles, interacting with each other). As described in the documentation we are using a setup-project to login in each of those users. While working on a new tests, to save the setup time, we are always comment out the unused users. Another use-case for this metadata is to centrally create new entities per test (in our case tasks) for each test that performs CRUD operations on it's own entity / task in isolation (e.g. using a batch creation REST-API).

It would be a dream if we could define meta-data (like e.g. the usernames used) for each test and access this information in the setup-phase. Only the information of actually to be executed tests (taking into account .only, .skip, cli-filter, ...) could then be used to only perform the login for users needed in the test.

Example

// my-setup.setup.ts
test.parallel("Setup users", ({ page, metadata }) => {
   const userList = metadata.testToExectute.map(meta => meta.annotation["users"])
   
   for (const userName of new Set(userList)) {
    test(`authenticate as ${userName}`, async ({ page }) => {
      test.slow();
      await loginAs(userName, page);
    });
  }
});


// my-sample.spec.ts
test.describe( "Order process", {
    tag: "@am",
    annotation: [
      { type: "users", description: "admin, clerk-a, clerk-b, editor-a" },
      { type: "entity", description: "sample-task-b" },
    ],
  },
  ({ page }) => {
    page.goto("...");
    // ...
  }
);

Motivation

It would allow to drastically speed up setups before the actual tests are executed. This is especially handy during development of new tests were we first want to focus on getting the test to run.