playwright icon indicating copy to clipboard operation
playwright copied to clipboard

[Question] Test execution speed for chromium vs stable browser (msegde)

Open Nav-2d opened this issue 2 years ago • 2 comments
trafficstars

I have a sample project that is created using npm init playwright@latest and trying to run the example.spec.ts on chromium vs stable edge (channel: 'msegde').

Observation:

  • Chromium execution takes ~1.8s
  • Stable Edge execution takes ~18s (almost 10x)

Is the above execution speed expected? Even opening the DevTools on headed is so slow. FWIW, I am on Mac M1 Pro (13.3.1).

Nav-2d avatar May 17 '23 20:05 Nav-2d

@Nav-2d Any chance for some policies/extensions/etc. in your Edge browser? This might explain performance issues. See the large danger notice in our docs.

If that's not the case, try recording a trace and explore it to see what 'is actually slow.

dgozman avatar May 17 '23 21:05 dgozman

@dgozman - Thanks for looking at it. I read the danger notice in the docs and I have no policy set in the browser.

The trace file shows the initial navigation to https://playwright.dev takes about 10s.

Attaching trace for reference. trace-1-edge.zip trace-edge.zip

Repo: https://github.com/Nav-2d/slow-stable-browser-pw

Nav-2d avatar May 17 '23 22:05 Nav-2d

@Nav-2d Unfortunately, the trace did not reveal much. It looks like all the operations are very slow, like if the CPU is overbooked. Not sure how to help here.

dgozman avatar May 19 '23 17:05 dgozman

@dgozman - Thanks for the reply. What I can do is I can try running the same on my windows laptop to see if there is any difference and post the findings here. Also, if I can request you to try it locally with stable browser when you have some bandwidth.

Appreciate the help

Nav-2d avatar May 19 '23 17:05 Nav-2d

Also, if I can request you to try it locally with stable browser when you have some bandwidth.

On my M1 Mac 13.3.1, there is no difference in headless mode, and ~2x slowdown in headed mode. So, nowhere near 10x slowdown. Edge version is 113.0.1774.50, playwright version is v1.33.

dgozman avatar May 19 '23 20:05 dgozman

Thanks Dmitry. On my Windows, it is working fine and they are about the same speed. So something up with my macbook. I will close the issue and update it later if I find/narrow it down to something specific.

Thank you for your time.

Nav-2d avatar May 19 '23 20:05 Nav-2d

Maybe I have narrowed it down. When I launch edge using: ...devices['Desktop Edge'], channel: 'msedge',. It tries to launch MS Edge which is Intel kind (as seen in Activity Monitor) instead of Apple kind.

Tried same scenario against a co-workers Mac and it is launching Apple kind and speed is fine.

Will try to figure out why it is picking up the Intel based.

Screenshot 2023-05-19 at 2 10 54 PM

Nav-2d avatar May 19 '23 21:05 Nav-2d

@Nav-2d Playwright bundled Chromium has only arm64 code on M1 whereas MS Edge has both x86_64 and arm64 (you can see that by running file /Applications/Microsoft\ Edge.app/Contents/MacOS/Microsoft\ Edge). If the terminal where you run the tests from or Node.js build is x86_64 and runs under Rosetta it will cause all subprocesses to run under Rosetta as well.

To figure out where it switches to Rosetta try running

arch
file $(which node)

in the terminal where you run the tests, it may give you a clue.

yury-s avatar May 22 '23 16:05 yury-s

thank you so much @yury-s. You were right it was running under Rosetta and launch x86_64 instead of arm64. It is resolved now

Nav-2d avatar May 22 '23 18:05 Nav-2d