How to properly authenticate in e2e-test-utils-playwright?

[chefranov]

Background

I'm trying to use the @wordpress/e2e-test-utils-playwright package to automate testing with Playwright, but I'm facing issues with authentication.

Issue

I've set up environment variables for authentication:

WP_BASE_URL=https://example.com/en/
WP_USERNAME=admin
WP_PASSWORD=123456

However, when I run my tests, no authentication happens automatically. When observing the test execution, I can see the browser opens the WordPress login page but doesn't fill in any credentials.

My implementation attempts

Attempt 1: Using built-in admin fixture

import { test, expect } from '@wordpress/e2e-test-utils-playwright';

test('Test with automatic authentication', async ({ admin, page }) => {
  // Authentication should be done automatically here
  await admin.visitAdminPage('edit.php');
  
  // Test fails here because we're redirected to login page
  await expect(page.getByText('Hello, admin')).toBeVisible();
});

According to what I understand from the documentation, using the admin fixture should automatically handle the authentication process using the environment variables I've set up. However, this is not happening.

Attempt 2: Custom fixture using RequestUtils

I also tried creating my own custom fixture with RequestUtils:

import { test as base, RequestUtils } from '@wordpress/e2e-test-utils-playwright';

export const test = base.extend<{
  login: () => Promise<void>;
}>({
  login: async ({}, use) => {
    await use(async () => {
      const requestUtils = await RequestUtils.setup({
        user: {
          username: process.env.WP_USERNAME!,
          password: process.env.WP_PASSWORD!,
        },
        baseURL: process.env.WP_BASE_URL,
      });
      await requestUtils.login();
    });
  },
});

Then using it in tests:

import { expect } from '@playwright/test';
import { test } from './custom-fixtures';

test('Should access admin dashboard', async ({ page, login }) => {
  // Try to authenticate
  await login();
  
  // Navigate to admin page
  await page.goto(process.env.WP_BASE_URL + '/wp-admin/');
  
  // Test fails here because we're still on the login page
  await expect(page.locator('#wpadminbar')).toBeVisible();
});

Questions

1. What's the correct way to authenticate in Playwright tests using @wordpress/e2e-test-utils-playwright?
2. Is there something missing in my implementation?
3. Could you provide a working example of authentication that automatically logs in before test execution?
4. Does the RequestUtils.login() method update browser cookies, or is it just for API requests? How do I get the authentication state applied to the browser context?

I would greatly appreciate a proper example of how to set up authentication that works correctly with this package. Thank you!

[talldan]

What's the correct way to authenticate in Playwright tests using @wordpress/e2e-test-utils-playwright?

From what I understand, using environment variables is the correct approach. Have you tried logging out process.env.WP_USERNAME / process.env.WP_PASSWORD to see if they're set correctly?

In the gutenberg repo, I believe the default values are used from here, so there's not anything special happening: https://github.com/WordPress/gutenberg/blob/eea73800953b713a462384733eb867be7326ab9b/packages/e2e-test-utils-playwright/src/config.ts#L1-L5

You probably need to look at a different example. Jetpack does use non-default credentials, to do this it sets properties on the process.env object rather than setting them on the command line because it pulls the values in from a config file at runtime: https://github.com/Automattic/jetpack/blob/08c26bc001bb9e297b6ce1b58169bd1e3000e474/tools/e2e-commons/helpers/utils-helper.js#L213-L223

That function is called from the file that defines the playwright config: https://github.com/Automattic/jetpack/blob/08c26bc001bb9e297b6ce1b58169bd1e3000e474/tools/e2e-commons/config/playwright.config.default.mjs#L45

Possibly you can replicate something like this, setting the env values to hardcoded strings as part of your playwright config to be sure it works.

Does the RequestUtils.login() method update browser cookies, or is it just for API requests? How do I get the authentication state applied to the browser context?

I believe it's just for API requests. Here's a link to how that gets setup for gutenberg's e2e tests (as part of the globalSetup) file that can be defined in the playwright config: https://github.com/WordPress/gutenberg/blob/trunk/test/e2e/config/global-setup.ts

Jetpack's implementation is pretty much identical.

[chefranov]

From what I understand, using environment variables is the correct approach. Have you tried logging out process.env.WP_USERNAME / process.env.WP_PASSWORD to see if they're set correctly?

Yes I tried and it shows what I set (WP credentials). So it's correct.

I tried also like this:

import { test, expect, RequestUtils, Admin } from '@wordpress/e2e-test-utils-playwright';

test('Admin logs into the admin panel and sees the settings', async ({ page, pageUtils, editor }) => {

    console.log('Username:', process.env.WP_USERNAME, ' / ', 'Password:', process.env.WP_PASSWORD); // Got correct creds

    // Prepare RequestUtils and log in as admin
    const requestUtils = await RequestUtils.setup({
        user: {
            username: process.env.WP_USERNAME!,
            password: process.env.WP_PASSWORD!,
         },
    });
    await requestUtils.login();

    // Use Admin for navigation in the admin panel
    const admin = new Admin({ page, pageUtils, editor });
    await admin.visitAdminPage('options-general.php'); // ← Error: Not logged in

    // Verify that we are on the correct page
    await expect(page).toHaveURL(/wp-admin\/options-general\.php/);
    await expect(page.locator('h1')).toContainText('Settings');
});

Maybe @gziolo can help with it. I see he has many commits to the package.

[gziolo]

Maybe @gziolo can help with it. I see he has many commits to the package.

I don't think I contributed to the package with Playwright utils. I don't have insights into how it works outside of Gutenberg. You might want to check how it is consumed in WordPress Core here: https://github.com/WordPress/wordpress-develop/tree/trunk/tests/e2e.

[talldan]

Sadly I don't think many of the original devs contribute any more.

When observing the test execution, I can see the browser opens the WordPress login page but doesn't fill in any credentials.

I guess this is from the WordPress redirect when logged out rather than the test trying to login.

Perhaps you're right, and requestUtils.login performs the authentication. I can't see anywhere else that it happens. Having said that, I don't think you need to call await requestUtils.login(); explicitly, it's already called as part of request utils setup.

I think maybe try mimicking those examples for setting up requestUtils is the next thing to try.

[talldan]

I found another repo that also uses the package that might serve as a guide - https://github.com/IndiraBiswas/playwright-woo-Commerce

edit: Though it also seems to use some functions that don't exist 😕

[andreiglingeanu]

I faced exactly the same issue now when playing with @wordpress/e2e-test-utils-playwright. The secret to getting the admin to automatically login is to properly pass all the use params in the playwright.config.js file. You must get everything from @wordpress/scripts/config/playwright.config, otherwise and most importantly pass use.storageState and use.baseURL. Without this, the requestUtils.login() will be lost.

You could opt out of logging the user in by emptying out the storageState in the test.use, like here: https://github.com/WordPress/gutenberg/blob/de11ef92c9f6478fedb7bd5f56b1612943d5c7a9/test/performance/specs/front-end-block-theme.spec.js#L16

I hope this helps you!