[Bug]: Starting local server from Playwright is taking longer testing time

[Xinran-Ma]

Version

1.42.1

Steps to reproduce

Pre-start local server:

1. run npm start and wait until server is ready for requests
2. run npx playwright test until the test is done

Cold-start local server:

1. run npx playwright test until the test is done

Expected behavior

The time spend on page.goto(xxx) remain similar.

Actual behavior

The 'cold-start' is taking way longer time to run page.goto than the 'pre-start'.

Cold-start:

Pre-start:

Additional context

The only solution we found so far is to increase PlaywrightTestConfig -> timeout from 30s to 60s to avoid single test reach out 30s time out.

Here's our current Playwright configuration:

const config: PlaywrightTestConfig = {
  webServer: {
    command: 'npm run start',
    /* Explicitly set url so the tests will wait for dev server to be available */
    url: 'http://localhost:3000',
    timeout: 300 * 1000,
    reuseExistingServer: !process.env.CI
  },
  testDir: './tests',
  /* Maximum time one test can run for. */
  timeout: 60 * 1000,
  expect: {
    timeout: 5000,
    toHaveScreenshot: {
      threshold: 0.1,
      maxDiffPixelRatio: 0.01,
      maxDiffPixels: 10
    }
  },
  /* Run tests in files in parallel */
  fullyParallel: true,
  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: !!process.env.CI,
  /* Retry on CI only */
  retries: process.env.CI ? 1 : 0,
  /* By default playwright will use 50% of available cores. */
  workers: process.env.CI ? '100%' : undefined,
  /* Reporter to use. See https://playwright.dev/docs/test-reporters */
  reporter: [['list'], ['html'], ['json', { outputFile: 'test-results.json' }]],
  /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
  use: {
    headless: true,
    actionTimeout: 0,
    baseURL: 'http://localhost:3000',
    /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
    trace: 'on-first-retry',
    /* Capture screenshots after each test */
    screenshot: 'on',
    /* Record video for each test */
    video: 'on',
    /* Grant access when writing clipboard */
    permissions: ['clipboard-write']
  },

  /* Configure projects for major browsers */
  projects: [
    {
      name: 'chromium',
      use: {
        ...devices['Desktop Chrome']
      }
    }
  ],
  /* Folder for test artifacts such as screenshots, videos, traces, etc. */
  outputDir: 'test-results/'
}

Environment

System:
    OS: Windows 10 10.0.19045
    CPU: (12) x64 Intel(R) Core(TM) i7-10810U CPU @ 1.10GHz
    Memory: 16.51 GB / 31.78 GB
  Binaries:
    Node: 18.7.0 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.19 - C:\Program Files\nodejs\yarn.CMD
    npm: 8.15.0 - C:\Program Files\nodejs\npm.CMD
  IDEs:
    VSCode: 1.88.1 - C:\Dev\Microsoft VS Code\bin\code.CMD
  npmPackages:
    @playwright/test: ^1.42.1 => 1.42.1

[dgozman]

@Xinran-Ma This is indeed surprising. Unfortunately, we won't be able to help unless you share a repro with us, so that we can run it locally.

[Xinran-Ma]

@Xinran-Ma This is indeed surprising. Unfortunately, we won't be able to help unless you share a repro with us, so that we can run it locally.

Yeah good idea. Will try to put my code in a public repo then share here.

[mxschmitt]

I'll close it in the meantime as part of the triage process feel free to re-file if you have a repro which we can run locally, thanks!