kit icon indicating copy to clipboard operation
kit copied to clipboard

Tests for the "writes" app produce side effects on disk

Open hd-4 opened this issue 1 year ago • 0 comments

Describe the bug

When tests run in packages/kit/test/apps/writes/, the route double-mount/+page.svelte is sometimes empty on disk after the tests finish. The tests pass on the run that leaves the file modified, but fail on future runs.

Reproduction

Run pnpm test a few times from packages/kit/test/apps/writes/. I wrote a quick fish script that usually shows the issue in about 10 runs:

cd packages/kit/test/apps/writes
for x in (seq 100)
  echo "Running test $x..."
  pnpm test &> /dev/null
  if git status | grep clean &> /dev/null
    echo "Working directory is clean after run $x."
  else
    echo "Working directory is dirty after run $x!"
    break
  end
end

Logs

No response

System Info

System:
    OS: macOS 13.2.1
    CPU: (8) arm64 Apple M1 Pro
    Memory: 907.83 MB / 16.00 GB
    Shell: 3.5.1 - /opt/homebrew/bin/fish
  Binaries:
    Node: 19.3.0 - /opt/homebrew/bin/node
    npm: 9.2.0 - /opt/homebrew/bin/npm
  Browsers:
    Chrome: 113.0.5672.126
    Firefox: 113.0.2
    Safari: 16.3
  npmPackages:
    svelte: 3.58.0 => 3.58.0
    vite: ^4.2.1 => 4.2.1

Severity

annoyance

Additional Information

I found the bug because I'm writing similar tests in another project and wanted to see how SvelteKit does it. I'm new to Playwright, but I think a possible cause is that Playwright runs multiple processes which execute the tests in parallel, leading to filesystem conflicts. The tests start with test.describe.configure({ mode: 'serial' });, but I think that controls how tests are run within each process rather than reducing the number of processes overall like the workers config does.

I'm not sure how the file would end up empty after the conflict.

hd-4 avatar May 26 '23 03:05 hd-4